DOCUMENT RESUME 



ED 436 419 



SE 063 000 



AUTHOR 

TITLE 



INSTITUTION 
SPONS AGENCY 

REPORT NO 
ISBN 

PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Mitchell, Julia H.; Hawkins, Evelyn F. ; Stancavage, Frances 
B.; Dossey, John A. 

Estimation Skills, Mathematics - in-Context , and Advanced 
Skills in Mathematics: Results from Three Studies of the 
National Assessment of Educational Progress 1996 Mathematics 
Assessment . 

American Institutes for Research, Washington, DC. 

National Center for Education Statistics (ED) , Washington, 
DC. 

NCES-200 0-451 
ISBN-0-16-050209-8 
1999-00-00 
283p . 

Education Publications Center (Ed Pubs), U.S. Dept, of 
Education, P.O. Box 1398, Jessup, MD 20794-1244. Tel: 
877-433-7827 (Toll Free) . For full text: 

<http : //nces . ed . gov/nationsreportcard> . 

Reports - Research (143) 

MF01/PC12 Plus Postage. 

Elementary Secondary Education; ^Estimation (Mathematics) ; 
^Mathematics Achievement; Mathematics Education; * Problem 
Solving 

^National Assessment of Educational Progress 



ABSTRACT 



This report presents information from three special studies 
conducted as part of the National Assessment of Educational Progress (NAEP) 
1996 mathematics assessment. It is intended primarily for mathematics 
educators and others concerned with mathematics education, such as curriculum 
specialists, teachers, and university faculty in schools of education. The 
three studies reported here were designed to provide greater detail on how 
students perform on particular types of mathematics questions. Studies 
include the Estimation Study, the Study of Mathematics- in-Context , and the 
Study of Students Taking Advanced Courses in Mathematics. The first study was 
designed to explore students' skills in estimation and was implemented at 
three grade levels. It concludes that although there has been significant 
improvement in mathematics performance overall since 1990 at all grade 
levels, the trend for student performance in estimation over the six years 
since the inception of the Estimation Study of 1990 is less clear. The second 
study was designed to assess problem-solving abilities within contexts that 
allow students to make connections across mathematics content areas. The 
Advanced study was administered at grades 8 and 12 and was designed to 
provide students who were taking or had taken advanced courses in mathematics 
an opportunity to demonstrate their full mathematical proficiency. (Contains 
71 tables and figures.) (ASK) 
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THE NATION S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally representative and 
continuing assessment of what America’s students know and can do in various subject areas. Since 1969, assessments have been 
conducted periodically in reading, mathematics, science, writing, history/geography, and other fields. By making objective information on 
student performance available to policymakers at the national, state, and local levels, NAEP is an integral part of our nation’s evaluation 
of the condition and progress of education. Only information related to academic achievement is collected under this program. NAEP 
guarantees the privacy of individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of 
Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through competitive 
awards to qualified organizations. NAEP reports directly to the Commissioner, who is also responsible for providing continuing 
reviews, including validation studies and solicitation of public comment, on NAEP’s conduct and usefulness. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for 
NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the National 
Education Goals; for setting appropriate student performance levels; for developing assessment objectives and test specifications 
through a national consensus approach; for designing the assessment methodology; for developing guidelines for reporting and 
disseminating NAEP results; for developing standards and procedures for interstate, regional, and national comparisons; for 
determining the appropriateness of test items and ensuring they are free from bias; and for taking actions to improve the form and 
use of the National Assessment. 
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This report presents information from three special studies conducted as part of the National 
Assessment of Educational Progress (NAEP) 1996 mathematics assessment. For more than a 
quarter of a century, NAEP has been the only nationally representative and continuing 
assessment of what students in the United States know and can do in various academic subjects. 
Each NAEP assessment is guided by a framework that specifies important learning outcomes in 
that subject area; the 1996 mathematics framework was an enhancement of the one used for the 
mathematics assessments in 1990 and 1992. The goal of the new framework was to define a 1996 
assessment that would: (1) more adequately reflect current curricular emphases and objectives, 
and yet (2) maintain a connection with the 1990 and 1992 assessments to measure trends in 
student performance. 1 

In addition to the main NAEP assessment, NAEP periodically conducts special studies 
focused on areas of interest to educators and others. Topics for some of these studies arise as a 
result of how students performed on NAEP; others are generated simply from research questions 
about teaching, learning, and assessment of student achievement. This report focuses on studies 
in mathematics; special studies have also been conducted in, for example, reading and writing. 



LPmwjpaDscB aamdl A[wdln<Bm(E<B {fdDv° ttHacs McBjpciD^d 

This report is intended primarily for mathematics educators and others concerned with 
mathematics education, such as curriculum specialists, teachers, and university faculty in schools 
of education. The three studies reported here were designed to provide greater detail on how 
students perform on particular types of mathematics questions. They include: the Estimation 
Study; the Study of Mathematics-in-Context, which will be referred to as the Theme Study; and 
the Study of Students Taking Advanced Courses in Mathematics, which will be referred to as the 
Advanced Study. The Theme Study and the Advanced Study were administered for the first time 
in 1996. The Estimation Study, on the other hand, had been administered twice before, in 1990 
and 1992. 



1 For a more in-depth description of the 1996 mathematics framework and how it guided the development of cognitive items, 
see the following: National Assessment Governing Board (1996). Mathematics framework for the 1996 National Assessment of 
Educational Progress. Washington, DC: Author; Reese, C. M., Miller, K. E., Mazzeo, J., & Dossey, J. A. (1997 J. NAEP 1996 
mathematics report card for the nation and the states. Washington, DC: National Center for Education Statistics. 
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The first study was designed to explore students’ skills in estimation. It was implemented at three 
grade levels and was the only one of the studies that provided trend information. Findings from 
the Estimation Study include the following: 

O Although there has been significant improvement in mathematics performance overall 
since 1990 at all grade levels, the trend for student performance in estimation over the 
6 years since the inception of the Estimation Study in 1990 is less clear. 

O Student performance in Estimation at grades 4 and 12 was stronger in 1996 than 
in 1990. 

O Student performance in Estimation at grade 8 appears to be level across the 3 years of 
the assessment. 

The Theme Study was administered at three grade levels and was designed to assess 
problem-solving abilities within contexts that allow students to make connections across 
mathematics content areas. Findings from the Theme Study include: 

O At the fourth-grade level, with the exception of the first problem, most students 
attempted to answer the questions posed, even though large percentages produced 
responses that were scored as “incorrect.” Although not definitive, this may be 
evidence that the thematic context of the block of questions encouraged students’ 
attention to the task of solving problems, even ones that proved to be difficult for 
most students. 

O At grade 8, unlike grade 4, many students did not attempt to answer the more 
complex questions that required them to write explanations or apply concepts in 
problem settings. 

© The response rate to the Theme questions at grade 12 was somewhere between the 
rates observed for grades 4 and 8, with most questions being attempted by at least 
90 percent of the students. 

O At all grade levels, students appear to have difficulty with complex multistep problems, 
even those that require only simple computational skills at each step of the problem. 

O At all grade levels, many students seemed to lack the mathematical knowledge needed 
to solve problems. Other students, however, appeared to understand the underlying 
mathematics but provided incorrect or incomplete responses as a result of carelessness, 
inexperience in writing out solutions to problems, or confusion over the wording of 
the question. 

O At all grade levels, no positive relationship was seen between the frequency with which 
students engaged in writing a few sentences about how to solve a mathematics 
problem, or writing reports or doing mathematics projects, and student performance on 
the Theme blocks. 

12 
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The Advanced Study was administered at grades 8 and 12 and was designed to provide 
students who were taking or had taken advanced courses in mathematics an opportunity to 
demonstrate their full mathematical proficiency. Findings from the Advanced Study include: 

O Students participating in the Advanced Study differed from those who did not 
qualify for the study in that they tended to come from homes providing a stronger 
educational context, both in materials and in level of parental education. In addition, 
based on their participation in Title I programs or qualification for the federal 
Free/Reduced-Price Lunch program, fewer Advanced Study students appeared to 
come from low-income homes. 

O As would be expected, students at both grade levels who met the criterion for inclusion 
in the Advanced Study performed substantially better than other students on the main 
NAEP mathematics assessment. 

O The results show that Advanced Study questions were quite difficult, even for students 
who were taking the more challenging mathematics courses that were prerequisite for 
participation in the study. Overall performance, measured by average percentage 
correct, was 36 percent at grade 8 and 30 percent at grade 12. At both grade levels, 
moreover, most of these students were unable to solve problems that required two or 
three successive steps to achieve the desired result. 

O At grade 12, students who were currently taking mathematics or who were, or had 
been, enrolled in an Advanced Placement (AP) mathematics course outperformed 
students in the study who were not currently taking a mathematics course or who 
had not taken an AP course in mathematics. 
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To provide a context for the special studies that are the focus of this report, the following 
sections give additional information about the NAEP 1996 mathematics assessment and about 
the manner in which the design and execution of the special studies relate to the main 
mathematics assessment. 
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The NAEP mathematics framework encompasses three cross-cutting domains: a content domain, 
a domain of mathematical abilities, and a domain of mathematical power. The content domain 
has five strands: Number Sense, Properties, and Operations; Measurement; Geometry and Spatial 
Sense; Data Analysis, Statistics, and Probability; and Algebra and Functions. 2 The domain of 
mathematical abilities describes the nature of the knowledge or processes that are involved in 
successfully handling mathematical tasks or problems; it includes Conceptual Understanding, 
Procedural Knowledge, and Problem Solving. The domain of mathematical power refers to 
students’ ability to reason, to communicate, and to make connections of concepts and skills 
across mathematics strands, or from mathematics to other curricular areas. Figure 1.1 
summarizes the structure of the framework for the NAEP 1996 mathematics assessment. 




i — 

SOURCE: National Assessment Governing Board, Mathematics Framework for the 1996 National Assessment of 
Educational Progress. 



2 The content strand Number Sense, Properties, and Operations was called Numbers and Operations in the 1990 and 1992 
assessments. The content strand Geometry and Spatial Sense was called Geometry in the 1990 and 1992 assessments. 
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The development of the questions for the special studies, although guided by the 
1996 NAEP mathematics framework, naturally focused on the goal of each individual study. 
Questions for the Estimation Study cut across the five content strands, but the main intent was to 
assess estimation skills. Questions for the Theme Study tended to emphasize problem-solving 
abilities within the context of real-life types of experiences. Finally, questions for the Advanced 
Study tended to include more content in Algebra and Functions than did questions in the main 
NAEP assessment. 

In addition to cognitive achievement questions, student assessment booklets for the 
special studies contained blocks of background questions. The background questions asked 
students to provide information about themselves, their classroom instruction, and their 
motivation to expend effort on the assessment. Teachers and school administrators of students 
participating in NAEP also responded to background questionnaires. Teachers provided 
information about their education, professional careers, curricular practices, and instructional 
approaches, as well as the resources available to them for teaching mathematics. School 
administrators answered questions about school policies and practices. 



The NAEP 1996 mathematics assessment was conducted nationally at grades 4, 8, and 12. 

As mentioned earlier, both the Estimation Study and the Theme Study also were conducted at 
grades 4, 8, and 12, while the Advanced Study was conducted at grades 8 and 12 only. Students 
for the Estimation and Theme Studies were selected through the same sampling design as 
students for the main NAEP assessment and were representative of all U.S. public and nonpublic 
school students. 8 Students selected for the Advanced Study were representative of students who 
had taken, or were enrolled in, more advanced mathematics courses. Specifically, to qualify for 
the Advanced Study, eighth-grade students had to be currently enrolled in, or already have taken, 
first-year algebra or a more advanced course in mathematics; and twelfth-grade students had to 
be currently enrolled in, or already have taken, a pre-calculus or pre-calculus-equivalent course 
or a more advanced course such as calculus. 

Following the model of the main NAEP data collection, school administrators of students 
participating in the special studies were surveyed at all grade levels, but mathematics teachers of 
participating students only were surveyed at grades 4 and 8. The exception was the Advanced 
Study, which included surveys of grade 12 mathematics teachers. 



3 See Appendix A for detailed information on sample selection. 
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Student performance on NAEP assessments has been reported using a variety of measures. 
Results for the main NAEP mathematics assessment are reported using the NAEP composite 
mathematics scale, which summarizes performance across five separate subscales — one for 
each of the five content strands. In addition to the NAEP mathematics scale, results also are 
reported using the mathematics achievement levels as authorized by the NAEP legislation 4 and as 
adopted by the National Assessment Governing Board. The achievement levels are performance 
standards based on collective judgments about what students should be expected to know and to 
do. Viewing students’ performance from this perspective provides some insight into the adequacy 
of students’ knowledge and skills and the extent to which they achieved expected levels of 
performance. The Board reviewed and adopted the recommended achievement levels derived 
from the judgments of a broadly representative panel that included teachers, education 
specialists, and members of the general public. 

For each grade tested, the Board has adopted three achievement levels: Basic , Proficient , 
and Advanced. For reporting purposes; the achievement level cut scores for each grade represent 
the boundaries between four ranges on the NAEP mathematics scale: below Basic , Basic , 
Proficient , and Advanced. The generic policy definitions of the achievement levels are shown 
below in Figure 1.2. The text of the descriptions of expected mathematics performance at each 
achievement level at each grade can be found in the NAEP 1996 Mathematics Report Card. s 
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Policy Definition is of NAEP Achievement Levels 



THE NATION’S 




Basic 



This level denotes partial mastery of prerequisite knowledge and skills that 
are fundamental for proficient work at each grade. 



Proficient 



This level represents solid academic performance for each grade assessed. 
Students reaching this level have demonstrated competence in challenging 
subject matter, including subject-matter knowledge, application of such 
knowledge to real-world situations, and analytical skills appropriate to the 
subject matter. 



Advanced 



This level signifies superior performance. 



SOURCE: NAEP 1996 Mathematics Report Card for the Nation and the States. 



4 The National Education Statistics Act of 1994 requires that the National Assessment Governing Board develop “appropriate 
student performance levels” for reporting NAEP results. 

5 Reese, et al., (1997). op. cit. 
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The NAEP legislation requires that the achievement levels be used on a developmental 
basis until the Commissioner of Education Statistics determines, as the result of a Congressionally 
mandated evaluation by one or more nationally recognized evaluation organizations that the 
achievement levels are “reasonable, valid, and informative to the public.” Upon review of the 
available information, the Commissioner of Education Statistics agrees with the recent 
recommendation of the National Academy of Science that caution needs to be exercised in the 
use of the current achievement levels, since in the opinion of the Academy “... appropriate 
validity evidence for the cut scores is lacking; and the process has produced unreasonable 
results.” 6 Therefore, the Commissioner concludes that these achievement levels should continue 
to be considered developmental and should continue to be interpreted and used with caution. The 
Commissioner and the Governing Board believe that the achievement levels are useful for 
reporting on trends in the educational achievement of students in the United States. 
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None of the special study assessment questions contributes to the NAEP composite mathematics 
scale. However, in 1990, the first year the composite scale and its component subscales were 
used, a separate scale was established that summarized performance on questions used in the 
Estimation Study. Each scale was constructed separately, and the metrics of the scales are 
arbitrary. Therefore, although each scale ranges from 0 to 500 across the three grade levels 
assessed, it is not possible to conclude that a student who performed at level 300 on the 
estimation scale, for example, and one who performed at level 300 on the main mathematics 
scale had both mastered the same proportion of their respective content domains. 7 The value of 
the estimation scale, like the value of the composite scale, is that it allows for trend analysis 
across years as well as making it possible to report the results using achievement levels. 

The results from the Advanced Study and the Theme Study did not lend themselves to 
either the development of separate proficiency scales or equating to the main NAEP mathematics 
scales. 8 Consequently, the overall results from the Theme Study and the Advanced Study are 
reported simply in terms of the percentages of questions that students answered correctly. Student 
performance on individual items also is highlighted for each of these studies. 



6 Pellegrino, J. W., Jones, L. R., & Mitchell, K. J. (Eds.). (1999). Grading the nation’s report card: Evaluating NAEP and 
transforming the assessment of educational progress. Committee on the Evaluation of National and State Assessments of 
Educational Progress, Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education, 
National Research Council, (p. 182). Washington, DC: National Academy Press. 

7 In the initial year of use, each scale was set to have a mean of 250 and a standard deviation of 50. 

8 See Appendix A for more detail. 
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Each special study is presented in a separate chapter. The second chapter of this report describes 
the Estimation Study, the third chapter depicts the Theme Study, and the fourth chapter 
characterizes the Advanced Study. This report also includes two appendices. The first provides 
additional information on the procedural and technical aspects of these special studies, and the 
second includes standard error tables for the data presented in the body of the report. 
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Estimation is a process whereby one approximates, through rough calculations, the worth, 
size, or amount of an object or quantity that is present in a given situation. The approximation, 
or estimate, is a value that is deemed close enough to the exact value or measurement to answer 
the question being posed. Beginning with recommendations issued in 1975, estimation has been 
seen as an important, even necessary, daily living skill that all students should have, whether 
they are working with paper and pencil, doing mental mathematics, or using technology. 1 
Estimation is now viewed as a necessary component of the school mathematics curriculum. 

Estimation in the school curriculum can range from making quick ad hoc estimates to 
applying “rules of thumb.” “Rule of thumb” operations include the application of mental 
checkmarks to a situation; for example, knowing that the circumference of a circle is slightly 
larger than 3 times the diameter. Other examples of estimation strategies include: 

O adding only the leading digits to approximate the sum of a long column of numbers 
that all have the same number of digits (e.g., approximating the sum of 
23 + 74 + 81 + 19 + 37 as 2 + 7 + 8 + 2 + 4 or about 23 tens, or 230); 

O using compatible numbers for proportional problems (e.g., approximating the number 
of cups of oats that are needed to make 12 cups of granola, given that 1% cups of oats 
are needed to make 5 cups, by reasoning that it takes about 2 cups of oats for 6 cups 
of granola, so it will take about 4 cups for 12); and 

O breaking or separating an uncommon shape into parts with known attributes to 
approximate a quality of the less common shape (e.g., approximating the area of 
an irregularly shaped region by separating it into 8 squares, each of which has an 
area of 4 square centimeters, and computing the area of the region as 8 x 4, or 
32 square centimeters). 2 



1 National Advisory Committee on Mathematical Education. (1975). Overview and analysis of school mathematics: Grades 
K-12 . Washington, DC: Conference Board of the Mathematical Sciences; National Council of Supervisors of 
Mathematics. (1978). Position paper on basic mathematical skills. The Mathematics Teacher, 71, pp. 147-152; National 
Council of Teachers of Mathematics. (1980). An agenda for action . Reston, VA: Author; National Council of Teachers of 
Mathematics. (1989). Curriculum and evaluation standards for school mathematics . Reston, VA: Author. 

2 Reys, B. (1988). Estimation. In Thomas R. Post (Ed.), Teaching mathematics in grades K-8. (pp. 262-284). Newton, 

MA: Allyn & Bacon, Inc. 
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The importance of estimation in the school curriculum was acknowledged in 1986 when 
the National Council of Teachers of Mathematics (NCTM) devoted its annual yearbook to the 
topic. 3 The NCTM also featured estimation as a central area within the Curriculum and 
Evaluation Standards in 1989. 4 Estimation is seen as a central mathematical process 
applicable across a wide range of mathematical content areas, including all five of the NAEP 
content strands: Number Sense, Properties, and Operations; Measurement; Geometry and 
Spatial Sense; Data Analysis, Statistics, and Probability; and Algebra and Functions. 
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It is because of the acknowledged importance of estimation that, since 1990, the national 
NAEP mathematics assessment has included blocks of questions focused on this topic. State 
data, however, were only collected in the 1992 Estimation Study. 5 The Estimation blocks 
include questions that require students to make estimates in a wide range of mathematical 
settings, from problems involving basic numeric operations to questions assessing students’ 
knowledge of chance and growth. Some questions also require students to make use of spatial 
estimation skills, such as looking at pictures of dots in a region or marbles in a jar and making 
numerical estimates of the total. 

Unlike the blocks in the main mathematics assessment or either of the other special 
studies, the Estimation blocks are administered using a paced audio tape. That is, students are 
paced through the Estimation Study questions by an audio tape that moves students along as 
they read and respond to the questions. This mode of administration is intended to minimize 
the time students have available to make actual computations or to give undue consideration to 
one question over another within the block. The test developers hope thus to increase the 
probability that students will actually use their estimation skills to respond to the questions. 
Furthermore, the paced audio tape reads the questions to the students, facilitating access to the 
problems for students with limited reading skills. 

Because of these differences in administration procedures, the 1996 Estimation blocks 
were presented to students in separate test booklets rather than being randomly spiraled with 
the blocks from the main assessment. There was a separate Estimation booklet for each grade 
level, each containing three blocks of cognitive questions. The first block consisted of 
non-estimation questions taken from the main assessment. The second block (also referred to as 
the trend block) was based on the Estimation block used for that grade level in 1990 and 1992. 
Although some questions in the trend blocks were modified to improve their measurement 
characteristics, care was taken to maintain a sufficient core of unmodified questions from 
previous assessments to ensure the integrity of the trend data. 



3 Schoen, H., & Zweng, M. (Eds.). (1986). Estimation and mental calculations: The 1986 Yearbook of the NCTM. Reston, 
VA: National Council of Teachers of Mathematics. 

4 National Council of Teachers of Mathematics. (1989). op. cit. 

5 Results from the state administration of the 1992 Estimation Study can be found in the following publications: Mullis, 

I. V. S., Dossey, J. A., Owen, E. H., & Phillips, G. W. (1993). NAEP 1992 mathematics report card for the nation and the 
states . Washington, DC: National Center for Education Statistics; National Center for Education Statistics (1993). Data 
compendium for the NAEP 1992 mathematics assessment of the nation and the states. Washington, DC: Author. 
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The third block in each booklet contained new estimation questions written for the 
1996 assessment. In contrast to the trend blocks, the new Estimation blocks utilized 
constructed-response as well as multiple-choice questions, and they also included questions 
that were designed to assess students’ ability to distinguish between situations in which it is 
more appropriate to over- or underestimate. In addition, the new blocks were written 
specifically for each grade level, whereas the trend blocks included a great deal of overlap 
across grades. As shown in Table 2.1, nearly half of the trend questions were used across all 
three grades, and, among the remaining questions, only those for grade 4 were unique. All of 
the questions in the new blocks, on the other hand, were unique to the grade level for which 
they were written. 




Level (Sj/n)(s7 IBlmk 



THE NATION’S 
REPORT 



CARO 






m 









Only 4 


Only 8 


Only 12 


4-8 


4-8-12 


8-12 


Trend 


10 


0 


0 


0 


10 


12 


New 


13 


15 


16 


0 


0 


0 


i : ~ - - - 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Unfortunately, unlike the Theme Study and Advanced Study questions, which are discussed in 
later chapters of this report, none of the Estimation Study questions have been released to the 
public. This is because the number of Estimation Study questions is limited, and NAEP 
expects to readminister them in future NAEP assessments in order to continue to track trends 
over time. A few mock questions, based on actual assessment questions, were written for this 
report to provide the reader with a sense of the kinds of questions included in the Estimation 
blocks. The following paragraphs and mock questions provide a description of the content of 
the Estimation Study. 
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As shown in Table 2.2, multiple-choice questions predominated at each grade level, 
despite the introduction of some constructed-response questions in the new blocks written for 
the 1996 assessment. 
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Multiple Choice 


Constructed Response 


Total 




30 

31 
38 


3 

6 

0 


33 

37 

38 




©KSCite © 






1 - — 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



The Estimation Study questions drew upon content from each of the five strands 
identified in the NAEP mathematics framework. However, as can be seen in Table 2.3, content 
from the Number Sense, Properties, and Operations strand and the Measurement 
strand predominated. 
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1996 EstiSm@tiGBi Q&esfmws, by Gmde Level md 
Content Stmnd 



THE NATION’S 





SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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In addition to questions in which the correct response was an actual estimate of the 
quantity posed, some questions asked students to indicate an appropriate method for arriving at 
an estimate, to identify the one response that could not satisfy the specified conditions, or to 
label possible responses as being either “reasonable” or “unreasonable” estimates. Some 
questions emphasized visual, or “sight” estimation, and a few required knowledge of whether 
over- or underestimation would be more prudent under a range of specified circumstances. In 
general, the questions designed for the higher grade levels exhibited increased complexity; for 
example, they required the combination of more facts or factors, involved more complex 
concepts or operations, or were presented in more general or abstract settings. 

Number Sense , Properties, and Operations 

Questions involving estimation that were classified in the Number Sense, Properties, and 
Operations strand required students to estimate quantities such as the following: 

o the result of a single mathematical computation; 

O the largest or smallest among the results of different arithmetic operations based on 
the same numbers; 

O a number rounded to a specified decimal place value; 

O the approximate total cost of several purchases; 

6 the total monetary value of several piles of coins; 

O the amount of material needed for several articles, given the amount of material 
needed for one; 

O the number of offspring after several cycles of reproduction; 

O the number of individuals in a subgroup, given information about the size of the total 
group and the percentage represented by the subgroup; 

O the mean score on a group of tests, given information about the number of tests and 
the minimum and maximum scores; 

O the number of persons present at the end of a sequence of events in which people 
both come and go; and 

O the number of words on a typewritten page, given an enlarged image of a portion of 
the page. 
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In addition to whole numbers, questions in this content strand involved fractions, 
decimals, money (including groups of coins), and time. At grade 12, exponents and roots also 
were included. A mock Number Sense, Properties, and Operations question similar to those 
that were asked of eighth-grade students is presented below. (As noted earlier, no Estimation 
Study items were released to the public, so the items shown in this chapter are not actual 
NAEP items.) 



As they were riding along a highway, Chris and Lee noticed that all of the cars in the 
opposite lane were stopped due to road construction. Driving at an average speed of 
48 miles per hour, it took Chris and Lee 11 minutes to get from the beginning of the 
stopped cars to the end. If the average length of a car is 20 feet, show how you could 
round the numbers and compute in order to estimate how many cars were stopped in 
the opposite lane. (1 mile = 5,280 feet) 

Not an operational NAEP question 

Ihis question is relatively difficult because it requires more than one operation or 
comparison to reach a solution, whether or not the computational burden is reduced through 
appropriate rounding. Students also need to understand the relationship between time, 
distance, and rate in order to arrive at a correct solution. 

One possible solution to this problem follows. However, because different students 
might approximate or round the values in different ways, credit must be based on appropriate 
identification and application of a solution strategy rather than a specific numeric outcome. For 
example, some students might approximate 11 minutes as 1/6 of an hour rather than 1/5 of an 
hour. Similarly, since the number of place values was not specified, the number of feet per mile 
might well be rounded up to 5,300 rather than down to 5,000. 

One possible solution 



48 mph is almost 50 mph and 1 1 minutes is about y hour, so the length of the 

stopped traffic is about 50 x y or 10 miles. Since 5,280 feet rounds to 5,000, the 

number of cars stopped is approximately 1212222), which is 2,500. 
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MeusmtriBmemt 

Questions about estimation that were classified in the Measurement strand involved estimates 
of quantities such as the following: 

O the length, in centimeters or inches, of an object shown to actual size; 

O the relationship between readings of numbers on two different vertical scales; 

O the number of objects that can be held in a one-quart container, given the number 
that fit into a one-cup container; 

O the number of smaller objects that could be drawn into the interior of a larger object 
of the same shape; 

O the length of a specified path between two points, given information about the shape 
of the path and the straight-line distance between the two points; 

O the time difference between travel on two different routes, given rates and graphs 
drawn to scale for each route; 

O the area of an irregularly shaped object, given a basic square unit of measure; and 

O the area of an irregularly shaped object, given information about the dimensions of 
two rectangles, one larger and one smaller than the object. 

A mock Measurement question similar to those that might have been asked of 
fourth-grade students is presented below. 



School 



O Kim' 



s House 




% 



Playground 



Kim rides her bicycle from her house to school, then from school to the playground, 
and finally back home. She travels a total of 12.1 miles. About how many miles 
apart are the playground and the school? 



CD 4 miles 
CD 5 miles 
CD 7 miles 
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The correct response is Option B, 5 miles. To answer the question, a student might read 
the graphic representation of Kim’s journey, subtract the mileage for each of the two labeled 
segments from the total mileage for the journey, and round the answer to whole miles. If a 
student rounded all of the mileage values before doing the computations, the same numeric 
answer would be obtained. Alternatively, a student simply might reason from the picture that 
the third side is slightly longer than the side measuring 4.2 miles; hence, the answer must be 
5 miles. (As noted earlier, all of the Estimation Study questions were presented via audio tape 
in order to establish the pace for the assessment and discourage students from undertaking 
precise calculations.) 

Geometry and Spatial Sense 

Only a few of the questions in the Estimation blocks tapped content classified as Geometry and 
Spatial Sense. Those that did required students to estimate the size of angles, given either a 
visual representation or information about the size of other angles in the same figure. 

Data Analysis 9 Statistics 9 and Probability 

Questions involving estimation that were classified in the Data Analysis, Statistics, and 
Probability strand all required the students to make estimates based on data presented in 
graphic or tabular format. Examples of the types of values they were asked to estimate include 
the following: 

• the approximate value of a data point on a graph; 

• the median of several values presented in a bar graph; 

® which of two politicians performed best overall, given a tabular display of votes 
earned in each of several election districts; and 

• the slope of the line that best fits data in a scatter plot. 

An example of a Data Analysis, Statistics, and Probability question similar to those that 
might have been asked of twelfth-grade students follows. 
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MONTHLY COSTS FOR TELEPHONE CALLING OPTIONS 
AT PHONEBELL TELEPHONE COMPANY 



Calling Option 


Cost per Month 


Call Blocking 


$4.04 


Call Forwarding 


$2.30 


Call Waiting 


$4.59 


Caller ID 


$6.55 


Caller ID Deluxe 


•• $7.50 



The table above shows the monthly costs for certain calling options offered by Phonebell 
Telephone Company. Stephan chooses the following options: call blocking, call waiting, 
and caller ID. Approximately how much would Stephan pay over a 6-month period for 
these options? 

CD $15 
CD $35 

CD $90 Not an operational NAEP question 

The correct answer is Option C, $90. Students would need to use the tabular data 
display to locate the monthly costs of the three specified options, sum them, and then multiply 
by six to obtain the 6-month total. These computations would be facilitated by using leading 
digits to sum the costs or by rounding the costs of the options before beginning. However, given 
the magnitude of the differences between response options, adept students would not have to 
perform any computations. They could see very quickly that only Option C was large enough to 
represent a plausible 6-month cost for three options, the cheapest of which costs more than 
$4.00 per month. 

Algebra and Functions 

Questions about estimation that were classified in the Algebra and Functions strand occurred 
only at grades 8 and 12. They required students to estimate quantities such as the following: 

O the cube root of a four-digit number; 

O the solution to a linear equation with decimal coefficients; and 

O the value of a trigonometric function of a fractional number of radians, given a graph 
of the function. 
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dDwetrmM performance on the NAEP mathematics scale 

The tables that follow contain information on student performance on the NAEP Estimation 
Study over the past three assessments. The first table, Table 2.4, also includes performance 
data from the main NAEP mathematics assessment. These data show that, for all three grades 
on the main mathematics assessment, there was a pattern of continuous improvement between 
1990 and 1992 and between 1992 and 1996 (although the improvement at grade 12 between 
1992 and 1996 was not statistically significant). The trend for student performance on the 
Estimation Study, however, is not so clear. In part, this may reflect the smaller number of 
estimation items in the assessment, and the correspondingly greater error of measurement. 

In any event, student performance in Estimation at grade 8 appears to be level across the three 
years of the assessment, while performance at grades 4 and 12 was stronger in 1996 than 
in 1990. 

In reviewing these data, it is important to remember that, because the Estimation 
Study was scaled separately from the main assessment, it is not appropriate to make direct 
comparisons of the average scale values obtained in a given year across the two scales. On 
the other hand, the Estimation Study scale, like the scale used for the main assessment, is a . 
cross-grade scale. This means that all three grades are portrayed on a single scale, and it is 
possible to obtain rough estimates of the performance differences across grade levels. 




Average Scale Scores for National NAEP 
and Estimation Studies, Grades 4, 8, and 12 
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Assessment Year 


Average Overall 
Scale Score in 
Mathematics NAEP 


Average Estimation 
Scale Score 


@05X33 Q 


1996 


224*t 


206* 






1992 


220* 


208* 




1990 


2 1 2 


200 
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1996 


272 *| 


270 




1992 


268* 


271 




1990 


263 


269 


@05X33 HIS 


1996 


304* 




297* 




1992 


299* 


294 


rz 


1990 


294 


292 



Significant difference from 1990. 
t Significant difference from 1992. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1996 Mathematics Assessments. 
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Other findings related to Estimmtiom proficiency mnd student 
chuructcristics 

Table 2.5 contains information from analyses made of the relationship between students’ 1996 
performance on the Estimation Study and their background characteristics. At grades 4 and 8, 
males and females performed about the same, while twelfth-grade boys outperformed 
twelfth-grade girls. At all three grades, White students and Asian/Pacific Islander students 
outperformed Black and Hispanic students. On average, students whose parents had at least 
some education after high school had higher scale scores than those whose parents did not 
finish high school, while at grades 4 and 12, students whose parents had a high school 
education also outperformed those who reported that their parents did not finish high school. 
Finally, students who did not participate in Title 1 and were not eligible for the federal 
Free/Reduced-Price Funch program performed better than those who did participate or were 
eligible for these programs. 

Data from earlier years are not included in Table 2.5 because there were few significant 
changes in subgroup performance in Estimation over the 6-year span from 1990 to 1996. The 
two significant changes that were observed were in the performance of males at grade 12 
(where performance increased from 296 in 1990 to 303 in 1996) and in the performance of 
fourth-grade White students (where performance increased from 208 in 1990 to 216 in 1996). 6 



6 The sources of these data are the 1990 and 1996 NAEP mathematics assessments. 
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Grade 4 


Grade 8 


Grade 12 


Gender 








Males 


209 


272 


303*t 


Females 


203 


268 


292 


Students who Indicated Their 
Race/Ethnicity as... 








White 


216* 


278 


303 


Black 


181 


246 


276 


Hispanic 


183 


253 


279 


Asian/Pacific Islander 


213! 


271 


318 


American Indian 


* * * 


* * * 


* * * 


Students who Reported Their 
Parents' Highest Level of 
Education as... 








Did Not Finish High School 


179 


257 


275 


Graduated From High School 


203 


262 


289 


Some Education After High School 


213 


275 


291 


Graduated from College 


217 


278 


306 


1 Don't Know 


197 


253 


* * * 


Students who Attend... 








Public Schools 


204 


269 


295 


Nonpublic Schools 


221! 


278! 


308! 


Title 1 Participation... 








Participated 


178 


248 


271! 


Did Not Participate 


217 


273 


298 


Free/Reduced-Price Lunch 
Program Eligibility... 








Eligible 


183 


254 


273 


Not Eligible 


217 


276 


299 


Information Unavailable 


218 


275! 


304 


r 









* Significant difference from 1990. 
t Significant difference from 1992. 

*** Sample size is insufficient ta permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may nat be accurately 
determined and/ar the sampling distribution af the statistics does nat match statistical test assumptions (see Appendix A). 
SOURCE; National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Table 2.6 presents data on students’ average Estimation scores at different percentile 
levels for 1990, 1992, and 1996. Here also, little change has occurred since 1990. At grade 
4, students at the median and above have made progress since 1990. No significant changes 
have taken place at grade 8. At grade 12, there has been some improvement for students at the 
low end of the distribution — the 10th percentile. These data indicate that the pattern of 
limited or no improvement noted earlier for performance on the Estimation Study questions is 
fairly widespread across the full range of student performance, with the possible exception of 
students at the upper end of the grade 4 distribution. 
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Average Scale Scores in Estimation at Different RE ™ R J 
Percentile Levels, Grades 4, 8, and 12 
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Assessment 


10th 


25th 


50th 


75th 


90th 




Year 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


















1996 


153 


180 


208 


235* 


256* 




1992 


160 


184 


210* 


234* 


253 




1990 


158 


178 


200 


223 


243 


















1996 


232 


251 


272 


290 


304 




1992 


233 


251 


272 


291 


306 




1990 


232 


249 


269 


288 


305 
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1996 




278 










259* 


298 


317 


333 




1992 


258* 


276 


295 


314 


328 




1990 


252 


273 


295 


313 


327 


I ' 



* Significant difference from 1990. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1 996 Mathematics Assessments. 
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Along with the development of the NAEP Estimation Study score scale, Advanced , Proficient , 
and Basic achievement levels also were established for the estimation questions. These 
achievement levels were first reported in 1990 and were used again, though in slightly 
modified form, in 1992 and 1996. 7 This section includes information on the percentages of 
students scoring at or above the achievement levels set on the Estimation Study questions. 

The data, presented in Table 2.7, reflect much the same picture as was seen in the 
earlier proficiency scale results — that is, there has been little change since 1990. The only 
area where a significant increase in student achievement appears to have taken place is at the 
fourth-grade level, where there was a significant increase from 1990 to 1996 in the percentage 
of students reaching at least the Proficient level of performance. 
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Assessment 

Year 


Advanced 


At or Above 
Proficient 


At or Above 
Sasic 


Below Basic 
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1996 


2 


30* 


88 


12 




1992 


1 


30* 


91 


9 




1990 


0! 


20 


90 


10 
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1996 


1 


19 


68 


32 




1992 


1 


20 


67 


33 




1990 


1 


18 


64 


36 
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1996 


6 


38 


83 


17 




1992 


4 


34 


82 


18 




1990 


4 


33 


79 


21 



[ 



* Significant difference from 1990. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1 996 Mathematics Assessments. 



7 Mullis, I. V. S., Dossey, J. A., Owen, E. H., & Phillips, G. W. (1991). The state of mathematics achievement : NAEPs 
1990 assessment of the nation and the trial assessment of the states. Washington, DC: National Center for Education 
Statistics; Mullis, et al., (1993). op. cit. 
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An overall analysis of the percentages of students reaching the various achievement 
levels in 1996 reflects a differentiated pattern across the grades. Grade 12 had the greatest 
percentage of students reaching the Advanced level (6%), while only one or two percent 
reached this level at the lower grades. The Proficient level was reached by almost one-third of 
the students at grades 4 and 12, while one-fifth of the eighth-grade students reached this level. 
At the same time, over 80 percent of the students at grades 4 and 12 reached at least the Basic 
level, compared to two-thirds of the grade 8 students. 

The data in Table 2.8 provide information parallel to that provided in Table 2.5. Here 
one can see the percentages of students in different subgroups reaching at least the Proficient 
level on the Estimation Study for each of the three grade levels in 1996. 

The data for 1990 and 1992 were not reported in the table due to the small number of 
significant differences in performance on estimation over the time period of the three 
assessments. However, at grade 4, male students, female students, White students, and students 
attending public schools all exhibited significant increases since 1990 in percentages reaching 
at least the Proficient level, indicating that this increase in proficiency occurred broadly across 
the largest subgroups of students. 8 



8 The sources of these data are the 1990 and 1996 NAEP mathematics assessments. 
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Grade 4 


Grade 8 


Grade 12 


Gender 








Males 


32* 


22 


47 


Females 


28* 


16 


31 


Students who Indicated Their 
Race/Ethnicity as... 








White 


38* 


25 


45 


Black 


9 


3 


13 


Hispanic 


1 1 


6 


18 


Asian/Pacific Islander 


38! 


16 


70 


American Indian 


16! 


6! 


18! 


Students who Reported Their 
Parents' Highest Level of 
Education as... 








Did Not Finish High School 


7 


6 


14 


Graduated From High School 


27 


10 


25 


Some Education After High School 


36 


19 


27 


Graduated From College 


40 


28 


51 


1 Don't Know 


21 


6 


18! 


Students who Attend... 








Public Schools 


28* 


18 


36 


Nonpublic Schools 


43! 


29! 


53! 


Title 1 Participation... 








Participated 


8 


2! 


7! 


Did Not Participate 


38 


21 


40 


Free/Reduced-Price Lunch 
Program Eligibility... 








Not Eligible 


38 


24 


39 


Eligible 


12 


6 


12 


Information Unavailable 


40 


24! . 


49 


I . . 









* Significant difference from 1990. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The overall results presented in this chapter, indicate a lack of significant progress over the 
6 years since the inception of the Estimation Study in 1990. In terms of the achievement 
expectations set for this area of mathematics, the greatest change appears to have taken place 
at grade 4, although smaller score increases were also found at grade 12. The differences in 
grade 4 performance may reflect action on the recommendation by the National Council of 
Teachers of Mathematics that emphasis be placed on estimation in the primary grades. 9 The 
lack of equivalent increases at the upper two grades may be an indication of the lack of focused 
instruction and opportunity to practice estimation concepts and skills in either classroom or 
assessment settings. 



9 National Council of Teachers of Mathematics. (1989). op. cit. 
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The Study of Mathematics-in-Context (also referred to as the Theme Study) was implemented 
to address specific reform recommendations regarding mathematics instruction and assessment. 
These recommendations assert that students benefit from mathematics instruction and 
assessments that enable them to make connections across mathematics content areas 
(e.g., Number Sense, Properties, and Operations with Geometry and Spatial Sense, or 
Measurement with both Geometry and Spatial Sense and Algebra and Functions) and to 
real-world contexts. 1 These instructional and assessment strategies also reflect reform efforts to 
include more integrated curriculum and project-based instruction in our mathematics classrooms. 



RM1ZEZP A[(MmnmnsiliraaM(iDm 

Students in grades 4, 8, and 12 who participated in the Theme Study were selected through 
the same procedures used in selecting students for the national NAEP mathematics assessment 
(also referred to as the main NAEP assessment). 2 Two 30-minute Theme blocks based on 
sustained real-life scenarios were constructed for each grade level. The use of a single 
real-world problem contexts for each Theme block was expected to engage students’ interest 
and sustain motivation throughout the assessment. Additionally, the 30-minute Theme blocks 
provided an opportunity to explore student responses to questions that were more detailed and 
complex than questions in the main NAEP assessment, in which all blocks were 15 minutes 
in length. 

Students participating in the Theme Study were each given an assessment booklet 
containing one of the Theme blocks for their grade level and a 15-minute, grade-appropriate 
block from the main NAEP assessment. In addition, the Theme Study booklets included blocks 
of background questions that also were administered to students in the main NAEP assessment. 



1 National Council of Teachers of Mathematics. (1989J, Curriculum and evaluation standards for school mathematics. 
Reston, VA: Author; Brullag, D. & Maples, C. (1992). Making connections: Beyond the surface. Arithmetic Teacher , 
85(3), 230-235. 

2 For information on sampling for the national NAEP mathematics assessment, see Appendix A and Allen, N. J., Jenkins, 
F., Kulick, E., & Zelenak, C. A. (1997). Technical report of the NAEP 1996 state assessment program in mathematics. 
Washington, DC: National Center for Education Statistics. 
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As with the main assessment, teachers of participating fourth- and eighth-grade students and 
principals from schools in which the fourth-, eighth-, and twelfth-grade students were enrolled 
also were surveyed. In addition, students were provided with calculators that they could use 
without restriction, and they were asked to indicate whether or not they had used the calculator 
when answering each question. 

Although all of the questions in the Theme blocks were centered around a single theme, 
a correct response to one question did not depend on correct responses to other questions in the 
block, and each question was scored independently. However, some questions did include two 
or more interdependent parts that were scored as a single unit. 

Unlike the Estimation Study or the main NAEP assessment, no achievement scale was 
developed for the Theme Study. Because the Theme Study was not intended to measure a 
unique aspect of mathematics achievement, a separate achievement scale was not appropriate. 
Moreover, the Theme Study booklets did not have enough questions per content strand 
(subscale) in common with the main assessment to allow for valid linking to the scale used in 
the main assessment. 



( JDwgjaamn^aailnciDm (ClhaajpilcBn 0 

The performance data presented in this chapter include block-level average percentage correct 
scores as well as average percentage correct scores for individual questions: for all students, 
and by gender and race/ethnicity subgroups where sufficient sample sizes allow. One Theme 
block from each grade was released for public use, and these released questions are discussed 
in detail, although student performance on the unreleased blocks also is referenced. 

Theme-block questions included mathematics content from more than one content 
strand and assessed a variety of mathematical abilities as defined by the 1996 NAEP 
Mathematics Framework. 3 Although assessment developers attempted to provide students with 
questions from an appropriate range of grade-level difficulty, there was no explicit attempt to 
make the two Theme blocks at each grade level (or across grade levels) equivalent in terms of 
overall difficulty level. Therefore, the reader should not make any formal comparisons of 
performance at different grade levels or across the two Theme blocks at any of the grade levels. 

The remainder of this chapter is divided into four sections, one for each grade level and 
one that summarizes the findings across grade levels. Each grade-level section begins with 
background information on student demographics and the prevalence of two specific 
instructional practices thought to be related to skills required for answering the Theme block 
questions. The instructional practices data are based on teacher responses to the following 
questions: (1) How often are students asked to write a few sentences about how to solve a 
mathematics problem? and (2) How often are students asked to write reports or do mathematics 
projects? It was hypothesized that answering a series of thematically-related mathematics 
questions might be a more familiar task for students who had carried out sustained mathematics 
projects, and that students who had experience writing about mathematics might perform better 



3 For information about the NAEP 1996 Mathematics Framework, see National Assessment Governing Board. (1996). 
Mathematics framework for the 1996 National Assessment of Educational Progress. Washington, DC: Author. 
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on questions asking students to explain how they arrived at their answers. The background and 
instructional information is followed by a presentation of overall student performance on each 
Theme block administered to that grade and, finally, by a discussion of each of the questions 
students encountered in the released Theme block. 



(BiraadhB 4 
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The data in Tables 3.1 and 3.2 show that the fourth-grade students who participated in the 
Theme Study were similar to the students who participated in the main NAEP assessment on a 
variety of demographic variables and on the two classroom practice variables. Nearly equal 
numbers of male and female students took each of the Theme blocks and the main NAEP 
assessment. The majority of students in each of the samples were White; very small 
percentages were Asian/Pacific Islander or American Indian. When asked about the highest 
level of education attained by their parents, students most often indicated that their mothers 
and fathers had “graduated from college.” However, more than a third of the students were 
unable to provide any information about their parents’ education. Nearly 90 percent of students 
attended public as opposed to nonpublic schools. About one-fourth of the students participated 
in the Title I program, and about a third were identified as being eligible for the federal 
Free/Reduced-Price Lunch program. 

The data in Table 3.2 show that fewer than 10 percent of the students in each of the 
samples had teachers who reported that students in their mathematics classes wrote a few 
sentences about solving a mathematics problem “nearly every day.” Teacher responses for the 
remainder of the students were distributed fairly evenly across the other response categories: 
“once or twice a week,” “once or twice a month,” and “never or hardly ever.” Teachers of these 
fourth-grade students reported asking students to write reports and do projects even less 
frequently than asking them to write a few sentences about solving a mathematics problem. 

For all samples, fewer than five percent had teachers who reported asking their students to 
write reports or do projects “once or twice a week” or more, and two-thirds or more had 
teachers who reported “never or hardly ever” asking their students to engage in such practices. 
However, the reader should keep in mind that reports and projects are typically long-term 
activities, and this factor may limit the frequency with which teachers could ask students to 
do them. 

Given these similarities between students in the Theme Study samples and those 
participating in the main NAEP assessment, it is reasonable to expect that fourth-grade 
students across the nation would have performed similarly on the Theme blocks to the students 
who actually participated in the Theme Study. 
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tlilain Assessment 


Theme Slock 1 
The Butterfly Booth 


Theme Bloch 2 
Recycling 










Gender 








Males 


51 


51 


53 


Females 


49 


49 


47 


Students who Indicated Their 
Race/Ethnicity as... 

White 


68 


68 


69 


Black 


15 


15 


14 


Hispanic 


13 


12 


13 


Asian/Pacific Islander 


3 


3 


3 


American Indian 


2 


2 


1 


Students who Reported Their 
Parents' Highest Level of 
Education as... 
Did Not Finish High School 


4 


4 


6 


Graduated From High School 


13 


12 


13 


Some Education After High School 


7 


7 


8 


Graduated From College 


40 


42 


38 


1 Don't Know 


36 


36 


35 


Students who Attend... 








Public Schools 


89 


87 


88 


Nonpublic Schools 


1 1 


13 


12 


Title 1 Participation... 
Participated 


22 


24 


23 


Did Not Participate 


78 


76 


77 


Free/Reduced-Price Lunch 
Program Eligibility... 

Eligible 


31 


33 


35 


Not Eligible 


53 


55 


54 


Information Not Available 

L 


16 


12 


1 1 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Theme Sloth 1 


Theme Slock 2 




Main Assessment 


The Sutteriily Sooth 


Recycling 










Students Whose Teachers Report 
Asking Students to Write a Few 
Sentences About How to Solve a 








Mathematics Problem... 








Nearly Every Day 


9 


7 


6 


Once or Twice a Week 


26 


28 


28 


Once or Twice a Month 


36 


35 


33 


Never or Hardly Ever 


29 


30 


33 


Students Whose Teachers Report 
Asking Students to Write Reports 








or Do a Mathematics Project... 








Nearly Every Day 


1 


0 


0 


Once or Twice a Week 


4 


2 


2 


Once or Twice a Month 


29 


27 


25 


Never or Hardly Ever 

i _ __ 


66 


70 


72 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The developers of the assessment questions for the two fourth-grade Theme blocks selected 
real-life contexts that should have been familiar to most, if not all, fourth-grade students in the 
United States. The released block involved setting up a Butterfly Booth at a school science 
fair; the unreleased block was about recycling. With each of these blocks, students were 
provided with a packet of materials to be used in answering some of the questions. The packet 
that accompanied the Butterfly Booth block contained a Butterfly Information Sheet, which 
featured beautifully colored illustrations of a Monarch butterfly, a Black Swallowtail butterfly, 
and a Common Blue butterfly; a 6-inch ruler marked with both inches and centimeters; and 
15 cut-outs of the Common Blue butterfly. The Butterfly Information Sheet also included 
information about the wingspan and height of the Monarch butterfly. Providing students with 
pictures of butterflies in such vivid, realistic colors may have made the task more interesting 
for many of the students. 
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BUTTERFLY INFORMATION SHEET 




Monarch Butterfly 



Black Swallowtail Butterfly 
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The questions in both blocks were similar in format. The six questions in the Butterfly 
Booth block were either short or extended constructed-response questions. Of the eight 
questions in the Recycling block, one was a multiple-choice question and the remainder 
were short or extended constructed-response questions. In both blocks, there were three or 
four questions in which students were asked to provide an explanation for how they arrived at 
their answer. 



dPwcBtt^aallll sttwdl(Bmtl 

Students overall performance, as measured by average percentage correct scores, is presented 
in Table 3.3. For both blocks of questions, students’ average percentage correct score was 
30 percent. Male and female students performed similarly; White, Hispanic, and Asian/Pacific 

i41 
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Islander students outperformed Black students, and White and Asian/Pacific Islander students 
also outperformed Hispanic students. The sample of American Indian students for both Theme 
blocks was too small to permit reliable estimates of their performance on the block as a whole 
or on individual questions. Therefore, the remainder of the discussion will not include 
information about the performance of American Indian students. 

Classroom practices, as measured by the frequency of writing a few sentences about 
how to solve a mathematics problem, or writing reports or doing mathematics projects, were not 
found to be related to student performance on either of the Theme blocks. 




THE NATION’S 



Average Percentage Correct Scores by Theme Block,, 

Grade 4, 1996 



IMMJp] 



e||: 





The Butterfly Booth 


Recycling 


@(H3£fe <3 






All Students 


30 


30 


Gender 






Males 


29 


30 


Females 


31 


29 


Race/Ethnicity 






White 


34 


33 


Black 


17 


20 


Hispanic 


21 


22 


Asian/Pacific Islander 


35 


35 


American Indian 


★ ★ ★ 


★ * * 


Students Whose Teachers Report Asking 
Students to Write a Few Sentences 
About How to Solve a 
Mathematics Problem... 
Nearly Every Day 


35 


34 


Once or Twice a Week 


30 


32 


Once or Twice a Month 


29 


27 


Never or Hardly Ever 


32 


30 


Students Whose Teachers Report Asking 
Students to Write Reports or Do a 
Mathematics Project... 
Nearly Every Day 


★ * * 


* * * 


Once or Twice a Week 


* * * 


* * * 


Once or Twice a Month 


30 


30 


Never or Hardly Ever 


30 


30 


\ 







*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The introduction to the block of questions about planning a Butterfly Booth for a school science 
fair is shown in Figure 3.1. The instructions included the following information about the 
expectations for acceptable responses: “In those questions where you must write an answer, it 
is important that your answer be clear and complete and that you show all of your work since 
partial credit may be awarded.” The instructions also informed students that “[s]ome questions 
may each require 5 minutes or more to think about and answer.” Thus, students were expected 
to take time to think about how to solve some of these problems. Finally, the manner in which 
the task of planning the booth was presented provided students the opportunity to take 
ownership of the task. For example, the introduction says, “Your class is planning to have a 
Butterfly Booth,” and “You need to make decorations for the booth... ” The hope was that 
students would feel that they were no longer just answering questions on another test but 
solving problems that they might encounter in their own lives at school, making the task of 
answering the questions more interesting and meaningful. 
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This part has 6 questions. Mark your answers in your booklet. You will 
have to fill in an oval or write your answer as directed. In those questions 
where you must write an answer, it is important that your answer be clear and 
complete and that you show all of your work since partial credit may be 
awarded. Some questions may each require 5 minutes or more to think about 
and answer. After each question, fill in the oval to indicate whether you used 
the calculator. 

Use the packet you have been given to help you answer the questions 
in this section. 




Each class in Oakville School will have a booth at the Science Fair. Your 
class is planning to have a Butterfly Booth. 

Your class has a lot to do to get ready for the Science Fair. You need to 
make decorations for the booth, plan activities, and order materials. 

n 
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Question! 1. Draw symmetrical figure. The first question that students encountered in the 
block was classified as a Geometry and Spatial Sense question and was designed to assess 
Problem Solving ability The question provided students with a context for the task they were 
asked to complete. That is, students could imagine rendering butterfly drawings that would be 
used to decorate the booth. 



1. The butterfly booth will be decorated with butterfly drawings. Draw 
only the missing markings on each picture to make each butterfly 
symmetrical. 




To respond successfully to this question, students needed to know what “symmetrical” 
meant, be familiar with using grids to complete drawings, and understand that the grids were 
important for getting a correct response. This question was scored with a 4-point rubric: 
“satisfactory,” “partial,” “minimal,” and “incorrect.” 4 To be scored “satisfactory,” responses 
had to have all four markings correctly drawn as shown in the sample student response. 
Students were given full credit if their drawings appeared to show that they understood the 
concept of symmetry, even if the symmetrical markings were not perfectly drawn or not placed 
exactly on the grid. 

SdamjpUcB rrcesjpGDmscB 




4 Student responses for this and all other constructed-response questions also were scored as “off task” if the student provided 
a response that was deemed not related in content to the question asked. There are many examples of this type of response, 
but a simple one would be “I don’t like this test.” In contrast, responses scored as “incorrect” were valid attempts to answer 
the question that simply were wrong. 
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A “partial” response was one in which three of the four markings were correctly drawn; 
the fourth marking was either omitted or incorrectly drawn. In the “partial” response that 
follows, the student appears to have understood the concept of symmetry but did not duplicate 
the fourth marking. The student may have decided that, because this marking had a less 
uniform shape, it was too difficult to attempt to draw. Another possibility is that the student did 
not have enough time to finish the task. 

66 ]pmv°ttn(mU 99 ^(BSjpdDmscs 




“Minimal” responses included those in which one or two of the markings were drawn 
correctly, and the remaining markings were either omitted or incorrectly drawn. The two 
“minimal” responses shown each have one marking correctly drawn. It is not clear why the 
students chose to draw only one marking. It is possible they did not fully understand the 
meaning of symmetrical. “Incorrect” responses had none of the parts correctly drawn. 
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Information on students’ performance on this question is presented in Table 3.4. 

The majority of students were able to draw at least some of the parts correctly; however, only 
28 percent of the students drew all four parts correctly. Furthermore, a relatively large 
proportion of students (33%) did not attempt to draw symmetrical butterflies at all; that is, they 
skipped over this problem entirely. It is not clear why so many students omitted this question; 
possibly they did not know the word “symmetrical” or did not know how to use the grid paper. 
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Satisfactory 


Partial 


fftlinimol 


Incorrect 


Omit 














All Students 


28 


17 


1 1 


1 1 


33 


Males 


30 


14 


8 


1 1 


36 


Females 


27 


20 


13 


10 


29 


White 


34 


19 


10 


10 


26 


Black 


8 


14 


13 


15 


50 


Hispanic 


18 


12 


1 1 


10 


48 


Asian/Pacific Islander 


38 


21 


8 


6 


27 


American Indian 


* * * 


* * * 


★ * * 


* * * 


* * * 



NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 2, Measure length using ruler. In the second question, students were asked to use 
the ruler that was provided in their packets to measure the wingspans of the Black Swallowtail 
and the Common Blue butterflies and report their measurements to the nearest centimeter. This 
question was classified as a Measurement question and was designed to assess Procedural 
Knowledge. Students needed to know how to measure using the ruler provided, what 
centimeters are, and what it means to measure to the nearest centimeter. Students also needed 
to know what the term “wingspan” means or be able to discern its meaning from the indication 
of the Monarch butterfly’s wingspan on the Butterfly Information Sheet. Furthermore, although 
centimeters are clearly indicated on the ruler provided, knowledge of and experience in using 
centimeter measurements may have been helpful. 



2. Take the Bnntterffly Information! Sheet from yom packet. 

On the Butterfly Information Sheet the wingspan of the Monarch 
butterfly is shown. 

Use your ruler to measure the wingspans of the other two butterflies on 
the sheet, the Black Swallowtail butterfly and the Common Blue 
butterfly, to the nearest centimeter. 

Black Swallowtail Wingspan: centimeters 

Common Blue Wingspan: centimeters 

The correct answer to the question is seven centimeters for the wingspan of the Black 
Swallowtail butterfly and three centimeters for the wingspan of the Common Blue butterfly. 
Students’ responses were scored on a 5-point rubric: “extended,” “satisfactory,” “partial,” 
“minimal,” and “incorrect.” An “extended” response was one in which both measurements 
were correct. Responses were scored “satisfactory” if one of the measurements was correct, if 
the two measurements were correct but reversed on the answer sheet, or if students did not 
round to the nearest centimeter but provided answers that would round to seven (6.5 to 7.5) and 
three (2.5 to 3.5). “Partial” responses were those in which students appeared to have measured 
the heights of the butterflies rather than their wingspans; that is, “partial” responses included 
answers for both measurements, with the wingspan ranging between 5 and 5.5 centimeters for 
the Black Swallowtail and between 2 and 2.5 centimeters for the Common Blue butterfly. 
Responses were scored “minimal” if they were in the correct range for inches, not centimeters. 
That is, if the measurements were 2.5 to 3 inches for the Black Swallowtail and 1 to 1.5 inches 
for the Common Blue. Responses that met none of the criteria already mentioned were scored 
as “incorrect.” 

Information on response scores for this question is presented in Table 3.5. Forty percent 
of fourth-grade students were able to provide the correct measurements for the wingspans of 
both butterflies. From student responses, it appears that many fourth-graders are familiar with 
metric measurements such as centimeters or are at least able to translate what they know about 
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using a ruler with inch markings into using a ruler with centimeter markings. Twenty-six 
percent of the responses were scored as “satisfactory” or “partial,” suggesting that these 
students may not have been careful in their measurements or in their reading of the question, 
may not have understood the term “wingspan,” or may not have understood what it means to 
measure to “the nearest centimeter.” 

Perhaps as evidence that fourth-grade students were more familiar with the task 
involved in this question compared to the first question, a much higher percentage of students 
chose to respond to this question (96%) than chose to respond to the first question (67%). 
Furthermore, the percentage of “extended” responses (40%) for this question was higher than 
the percentage of “satisfactory” responses (28%) for Question 1. 
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NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 
***$ample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 3. Solve packing problem. Students were asked to use the 15 rectangular cut-outs 
of the Common Blue butterflies that were provided in their packets to answer this two-part 
question. The question included a two-dimensional representation of a storage case, drawn to 
size. Part 1 of the question asked students to determine the greatest number of Common Blue 
butterflies (of the size of the cut-outs) that could be stored in the case and to show how those 
butterflies would be arranged in the case; butterflies could not overlap each other or be stacked 
on top of each other. In the second part of the question, students were asked to determine how 
many storage cases of the same size they would need to store 28 Common Blue butterflies. 
Students also were asked to explain with drawings, words, or numbers how they arrived at their 
answers. Question 3 was classified as a Measurement question and was designed to measure 
Problem-Solving ability. 

In responding to this question, students were not required to do any actual measuring 
with a ruler. To determine the number of butterflies that would fit, they simply needed to place 
the Common Blue butterfly cut-outs onto the picture of the storage case. To indicate how the 
butterflies would lie in the case, students could trace the outline of the cut-outs or they could 
draw butterflies of the approximate size of the Common Blue cut-outs. Having answered Part 1, 
there were numerous strategies students could use to solve Part 2. Whatever strategy students 
used, they were required to describe it in order to answer the question completely. 
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3. Take the butterfly cutouts from your packet. 

What is the greatest number of Common Blue butterflies that can be 
stored in the case below? (When you put butterflies in the case, you 
can't stack them. The butterflies can touch, but they can't overlap at 
all.) 

Answer: 

Show how the butterflies fit in the case. 

Storage Case 



12 centimeters 



(- 6 centimeters ~| 

How many storage cases would you need to store 28 Common Blue 
butterflies? 

Answer: 

Use drawings, words, or numbers to explain how you got your 
answer. 
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This question was scored on a 5-point rubric: “extended,” “satisfactory,” “partial,” 
“minimal,” and “incorrect.” Responses scored as “extended” provided the answer 12 for the 
number of butterflies that would fit in the case, with a drawing of how the 12 would lie in the 
case, and the answer 3 for the number of cases needed to accommodate 28 butterflies, with an 
appropriate explanation for arriving at 3 cases. In the sample “extended” response that follows, 
the student subdivided the picture of the storage case into 12 appropriately sized rectangles to 
show the placement of the 12 non-overlapping butterflies. For the second part of the question, 
the student showed, through addition, how she or he arrived at the answer of 3 cases needed to 
accommodate the 28 butterflies. 
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A response could be scored as “satisfactory” if it met any of the following criteria: 

O Part 1, 12 butterflies with a correct drawing; and Part 2, an incorrect answer but a 
correct explanation; 



O Part 1, 12 butterflies with a correct drawing; and Part 2, a correct answer but an 
inadequate or missing explanation; 

O Part 1, 12 butterflies with drawing missing; and Part 2, a correct answer with a 
correct explanation; or 

O Part 1, 10 or 11 butterflies with a correct drawing; and Part 2, a correct answer with 
a correct explanation. 
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The following sample “satisfactory” response includes the correct answers for Parts 1 
and 2 and an acceptable explanation for Part 2, but the drawing is missing for Part 1. It is not 
evident why the student did not complete the drawing. It is possible that she or he simply did 
not read the directions carefully and therefore did not realize that it was necessary to show how 
the 12 butterflies would fit in the case. 

SmmjpUtB v°<B§jpaDm§(B 

Answer: /J| § 1\&J> 

Show how the butterflies fit in the case. 

Storage Case 




53 



44 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



!§ffl mpfks 6S smttnsffm(S!t<]Dd 0 Q{i 99 Q°<esflD<iDtt ns© ((Eanmttnmwsdl) 



How many storage cases would you need to store 28 Common Blue 
butterflies? 

Answer: 

Use drawings, words, or numbers to explain how you got your 
answer. 
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A “partial” response would meet one of the following criteria: 

0 Part 1, 10 or 11 butterflies with or without a drawing; and Part 2, an incorrect, answer 
with a correct explanation of the strategy used to determine that number; 

0 Part 1, 10 or 11 butterflies without a drawing; and Part 2, a correct answer with an 
acceptable explanation; 



O Part 1, 10 or 11 butterflies with a correct drawing; and Part 2, a correct' answer but a 
missing or inadequate explanation; 

® Part 1, 12 butterflies without a drawing; and Part 2, an incorrect answer but an 
acceptable explanation; or 

© Part 1, 12 butterflies without a drawing; and Part 2, a correct answer but an 
inadequate or missing explanation. 

The sample “partial” response that follows indicated that 10 butterflies would fit in the 
case and provided a correct drawing of the 10 in the case. For Part 2, the student offered the 
correct answer of 3 cases but an inappropriate explanation of the solution process. The 
explanation in Part 2 is actually an explanation of deriving the answer in Part 1. 
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Answer: 



Show how the butterflies fit in the case. 



12 centimeters 



6 centimeters 



Storage Case 




How many storage cases would you need to store 28 Common Blue 
butterflies? 



Answer: <<J 

Use drawings, words, or numbers to explain how you got your 
answer. 
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The different responses that could be scored as “minimal” include the following: 

O Part 1, 10 or 11 butterflies with or without a correct drawing; and Part 2, an incorrect 
number of cases and explanation; 

O Part 1, 10 or 11 butterflies with an incorrect drawing; and Part 2, a correct answer 
but an explanation that is missing or inadequate; or 

O Part 1, 12 butterflies with or without a correct drawing; and Part 2, an incorrect 
answer and explanation. 

The sample “minimal” response that follows has an acceptable number of 10 butterflies 
but an incomplete drawing for Part 1 and the correct number of 3 cases but an inadequate 
explanation for Part 2. 
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Answer: 

Show how the butterflies fit in the case. 



12 centimeters 



f - — - 6 centimeters 



Storage Case 





How many storage cases would you need to store 28 Common Blue 
butterflies? 



Answer: 




Use drawings, words, or numbers to explain how you got your 
answer. 
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An “incorrect” response is one that meets none of the criteria previously mentioned. In 
particular, it provides no answer for the number of butterflies or an answer other than 10, 11, or 
12. The following “incorrect” response indicates that the greatest number of butterflies that 
would fit in the case is 6, and that 34 storage cases would be necessaiy to fit 28 butter-flies. 
Although the answer 6 is incorrect, the drawing for Part 1 shows some understanding of the 
task of explaining how the butter-flies would lie in the case. The answers to the second part of 
the question appear to indicate that the student did not understand at all what the question 
was asking. 
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Answer: 



Show how the butterflies fit in the case. 
Storage Case 




12 centimeters 
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Student performance information for this question is presented in Table 3.6. Students 
appear to have found this question difficult compared to the previous measurement question. 
Only a third of the responses were scored “partial” or higher, about a third were scored 
“minimal,” and just under a third of the responses were “incorrect.” Despite the apparent 
difficulty students encountered with this question, only one percent of the students chose not to 
attempt to answer it. It is perhaps both the multistep feature and the need to provide an 
explanation of the process of getting to an answer that accounted for much of the difficulty 
students had with this question. 
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NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

***$ample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 4. Oeterrame Member of models. The fourth question explained that students who 
visit the booth would be building models of butterflies. Students were told how many of each 
part of the butterfly — 4 wings, 1 body, and 2 antennae — were needed to build one butterfly. 
They were asked to determine how many complete butterflies could be built with the supply of 
29 wings, 8 bodies, and 13 antennae that the class had. Students also were asked to explain 
how they arrived at their answer using drawings, words, or numbers. This question was 
designed to assess Problem-Solving ability and content from the area of Number Sense, 
Properties, and Operations. 

To answer this question correctly, students needed to understand that, after building all 
possible complete butterflies, they might have remaining wings, bodies, and/or antennae. Their 
task, actually, was to determine for each part how many models of butterflies they could 
accommodate. That is, they had enough wings for 7 butterflies, enough bodies for 8 butterflies, 
and enough antennae for 6 butterflies. So, students had to know that the answer they were 
seeking was the smallest of those three numbers, namely, 6. 
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This question was scored using a 4-point rubric: “satisfactory,” “partial,” “minimal,” 
and “incorrect.” “Satisfactory” responses had the correct number of butterflies, 6, and a correct 
explanation for how that number was determined. The sample “satisfactory” response shown 
had a correct answer and a complete explanation. 

Sc mmjpUcB 66 sda{tnsff(ia(£tl(iD^ / Qi f 9 v°<BS]p<ibms<B 




Responses scored as “partial” either had the correct answer but an incomplete or 
erroneous explanation or had a correct explanation with the six-and-a-half pairs of antennae 
rounded to 7 butterfly models. The “partial” answer that follows had the correct answer 6 but is 
lacking an explanation. 
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Answer: 



Use drawings, words, or numbers to explain how you got your 
answer. 
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Responses scored as “minimal” had an incorrect number of butterfly models but 
provided some evidence that the student understood that the number of parts (wings, bodies, 
and antennae) available determined the number of complete butterfly models that could be 
built. The following sample “minimal” response has the wrong answer, 8, but the drawing of the 
butterflies shows an understanding of how the different parts are needed to build a complete 
butterfly model. 
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An “incorrect” response showed no understanding of how to solve the problem. For 
example, the following “incorrect” response shows that the student simply added the various 
available butterfly parts and came up with the sum, 50, which is the wrong number of complete 
models possible. 
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Student performance information on this question is presented in Table 3.7. Eighteen 
percent of the students were able to provide a response that received at least a “partial” score. 
Such responses appear to indicate that the student had some idea of how to solve the problem 
correctly. However, the majority of responses were scored as “incorrect,” indicating that many 
students found this question very difficult. 
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NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



65 




S© 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



Qeestiioii 5. Bettermme mimber of leaves. The fifth question used the context of the 
Butterfly Booth indirectly; that is, the question is about feeding caterpillars. Students could have 
answered this question correctly even if they did not know that butterflies are transformed 
caterpillars, but having that knowledge might have made the problem more interesting. In the 
question, students were told that a class has two caterpillars and needs five leaves a day to feed 
them. Students were asked to determine how many leaves the class would need each day if it 
had 12 caterpillars to feed. This question was classified as a Number Sense, Properties, and 
Operations question and was designed to assess Problem-Solving ability. As with other 
questions, in addition to specifying the number of leaves needed, students were asked to 
explain how they determined their answer. 

There were a number of strategies that students might have used to answer this 
question. For example, students may have reasoned that 12 is 6 groups of 2, so 6 times 
5 leaves, or 30 leaves are needed. 
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This question was scored on a 3-point rubric: “complete,” “partial,” and “incorrect.” 
Responses were scored “complete” if they had the right number of leaves, 30, and a correct 
explanation. The “complete” response that follows has the correct answer and an adequate 
computational explanation. Through the computations, the student appears to show an 
understanding that the number of caterpillars and the number of leaves needed to be multiplied 
by the same number, 6. 

Sample “ complete " response 




“Partial” responses either had the correct number of leaves with an incomplete, 
erroneous, or no explanation or a correct explanation with a wrong number of leaves because of 
a computational error. In the sample response below, the student has the correct number of 
leaves, 30, but the explanation is incomplete. The student started with an acceptable process of 
determining the number of leaves one caterpillar needs a day, that is, two-and-a-half, but then 
jumped to the conclusion of needing 30 leaves without explaining that 30 leaves is the answer 
to two-and-a-half leaves per caterpillar multiplied by 12 caterpillars. 

Sample “partial" response 



Answer: _ 30 

Use drawings, words, or numbers to show how you got your answer. 

fl&uSLrW 
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“Incorrect” responses had the wrong number of leaves and either an erroneous or no 
explanation. In the sample “incorrect” response shown below, the student forgot that the five 
leaves were for two caterpillars instead of one. The process explained was an acceptable 
process. It would have resulted in the correct answer had the student remembered that the five 
leaves were for a pair of caterpillars and therefore only counted by fives 6 times rather than 12. 
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Use drawings, words, or numbers to show how you got your answer. 
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Student performance information on this question is presented in Table 3.8. It proved 
to be a very difficult question for students; 86 percent of their responses were scored as 
“incorrect.” Despite the fact that students had difficulty determining the correct answer, most 
students attempted to answer the question. 
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NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 6. Interpret pattern of figuires. The last question in the Butterfly Booth block was 
about making a banner for the booth. Students were told that they should use the Butterfly 
Information Sheet to help them solve this problem. They were told the length of the banner, 

130 centimeters, and the design on the banner; that is, a repeating pattern of one Monarch 
butterfly followed by two Black Swallowtail butterflies with wings touching but not overlapping. 
Students were asked to determine how many Monarch and Black Swallowtail butterflies would 
be needed to fill the banner. This question was classified as an Algebra and Functions question 
and was designed to assess Problem-Solving ability. As with other questions, students were 
asked to explain how they got their answer. 
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This was a relatively complex question. Students first had to determine which of all the 
information available was needed to solve the problem and, second, to determine a multistep 
process for arriving at the number of Monarchs and Black Swallowtails they needed to complete 
the banner. As with Question 2, students who previously had conducted measurements using 
centimeters may have had an advantage in solving this problem. The correct response was 
6 Monarch and 10 Black Swallowtail butterflies and an adequate explanation of the process for 
obtaining that response. 

Although there are other strategies that students could have used to solve the problem, 
the following is one possible strategy: 

1. Measure in centimeters the wingspan of the Monarch butterfly and the Black 
Swallowtail butterfly — 10 centimeters and 7 centimeters. 

2. Add the wingspan measurements of one Monarch and two Black 
Swallowtails — 24 centimeters. 

3. Divide 24 centimeters into 130 centimeters, the length of the banner, to get the 
number of patterns needed to cover the banner — five patterns. 

4. Realize that there was 10 centimeters of banner remaining — enough to fit 
Monarch butterfly, but no additional Black Swallowtails. 

The responses to this question were scored on a 4-point rubric: “satisfactory,” “partial,” 
“minimal,” and “incorrect.” A “satisfactory” response had the correct number of Monarchs and 
Black Swallowtails and an adequate explanation of how the student arrived at those answers. 
The sample “satisfactory” response provides the correct answer and provides, as the 
explanation, a drawing of the how the butterflies (indicated with an “M” for Monarchs and a 
“W” for Black Swallowtails) would be positioned on the banner with the correct number of 
centimeters (for one Monarch and two Black Swallowtails) indicated below these letters. 






How many of each type of butterfly are needed for the banner? 
Monarch ^ 



Black Swallowtail 



Show how you got your answers. 
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“Partial” responses either had the correct number of Monarchs and Black Swallowtails 
but no explanation or an incomplete explanation, or had the correct strategy explained with the 
correct number of patterns but the wrong number of Monarchs and Black Swallowtails. The 
“partial” response shown had the wrong number of butterflies, but provided an explanation that 
showed an appropriate strategy for solving the problem. In the explanation, the student showed 
knowledge of the number of centimeters needed for the pattern, 24; had a counting strategy to 
get to the number of patterns; and understood that the banner could have an incomplete pattern 
at the end. Although it is not completely clear, it appears that the student made a mathematical 
calculation error while counting by 24 and came up with four repeating patterns with 
centimeters left over for the Monarch but no other Black Swallowtails. 
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How many of each type of butterfly are needed for the banner? 

Monarch S 

Black Swallowtail 

Show how you got your answers. 
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Responses that had any of the following pairs of numbers for Monarchs and Black 
Swallowtails with no or an inadequate explanation were scored as “minimal”: 4 and 8, 5 and 
8, 5 and 10, 6 and 12, or 7 and 12. These responses, which were classified as minimally 
acceptable, indicated measurement or computational error but showed that the student had 
some understanding of how to solve the problem. A sample “minimal” response follows. The 
student had a minimally acceptable number of repeating patterns, but did not provide any 
explanation and did not appear to take into consideration the remaining centimeters that could 
accommodate an additional Monarch butterfly. 
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How many of each type of butterfly are needed for the banner? 

Monarch 

Black Swallowtail 



Show how you got your answers. 
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Incorrect” responses involved pairs of numbers other than those mentioned above or 



had missing numbers. The following “incorrect” response shows an understanding of some 
aspects of the problem, but the student clearly did not understand that it was necessary to take 
into account the wingspans of all three butterflies in the pattern. Although it is not possible to 
know with certainty from the response how the student went about solving the problem, it 
appears the student took the wingspan measurement of the Monarch butterfly, 10 centimeters, 
and determined that 13 such butterflies could fit on a 130-centimeter banner by dividing 
130 by 10. Then, while not attending to the fact that the Black Swallowtails also have 
wingspans that take up space, the student laid out the repeating patterns with the 13 Monarchs. 
It is interesting, however, that the student did not add the two Black Swallowtails after the 
thirteenth Monarch butterfly. Therefore, the total number of Black Swallowtails summed to 24. 

Sumple “incorrect 99 response 



How many of each type of butterfly are needed for the banner? 

Monarch 

Black Swallowtail . 




Show how you got your answers. 
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Student performance data on this last question in the Butterfly Booth block are 
presented in Table 3.9. Students apparently found this question very difficult to solve: 
90 percent of the responses were scored as “incorrect.” 
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1 


Females 


i 


4 


4 


89 


1 


White 


2 


4 


4 


89 


1 


Black 


0! 


1 


5 


93 


1 


Hispanic 


0! 


1 


4 


91 


1 


Asian/Pacific Islander 


0! 


1! 


9 


87 


3 


American Indian 

i 


* * * 


* * * 


* * * 


* * * 


* * * 



NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

***$ample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Eighth-grade students who participated in the Theme Study were similar to eighth-grade 
students in the main NAEP assessment in terms of a variety of demographic characteristics. 
(See Table 3.10.) Students in grade 8 also were similar to students in grade 4. For example, 
there were similar percentages of male and female students; White students were in the 
majority; the modal response regarding parents’ highest level of education was “graduated from 
college.” In addition, about 90 percent of the students were from public schools; the large 
majority were not Title I students; and just over a quarter of the students were eligible for the 
federal Free/Reduced-Price Lunch program. 
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Klein Assessment 


Theme Block 1 
Building a Doghouse 


Theme Block 2 
Flooding 


©cods © 








Gender 








Males 


52 


53 


52 


Females 


• 48 


47 


48 


Race/Ethnicity 








White 


69 


70 


70 


Black 


14 


14 


14 


Hispanic 


12 


12 


12 


Asian/Pacific Islander 


3 


• 2 


2 


American Indian 


1 


2 


1 


Students who Reported Their 








Parents' Highest Level of 








Education as... 








Did Not Finish High School 


7 


8 


8 


Graduated From High School 


22 


23 


25 


Some Education After High School 


19 


18 


18 


Graduated From College 


42 


42 


39 


1 Don't Know 


1 1 


10 


9 


Students who Attend... 








Public Schools 


89 


90 


91 


Nonpublic Schools 


1 1 


10 


9 


Title 1 Participation... 








Participated 


12 


1 1 


10 


Did Not Participate 


88 


89 


90 


Free/Reduced-Price Lunch 








Program Eligibility... 








Eligible 


27 


25 


26 


Not Eligible 


55 


54 


54 


Information Not Available 


17 


21 


21 



[ 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The data in Table 3.11 show similar levels of exposure across the three samples of 
eighth-grade students to the classroom practice of writing a few sentences about how to solve a 
mathematics problem. Similarly, exposure to writing reports or doing mathematics projects did 
not vary significantly across the three samples of eighth-grade students. 





' THE NATION’S 




Perc@ntt@g@ m §9midleinitts by Tern heirs' ^ep@r&s @ev 
€Bassm@m Pmetikes, ©mde 8, B996 


i mt 


arc] 

1 : 







Main Assessment 


Theme Slock 1 
Building a Doghouse 


Theme Slock 2 
Flooding 


dtaac © 








Students Whose Teachers Report 








Asking Students to Write a Few 








Sentences About How to Solve 








a Mathematics Problem... 








Nearly Every Day 


5 


5 


4 


Once or Twice a Week 


25 


19 


21 


Once or Twice a Month 


37 


40 


36 


Never or Hardly Ever 


33 


37 


38 


Students Whose Teachers Report 








Asking Students to Write Reports 








or Do a Mathematics Project... 








Nearly Every Day 


0! 


0! 


0! 


Once or Twice a Week 


3 


4 


4 


Once or Twice a Month 


33 


34 


31 


Never or Hardly Ever 


64 


62 


65 



! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The two blocks of questions for the eighth-grade Theme Study also had interesting and relevant 
contexts. The released block involved building a doghouse; the unreleased block was based on 
the flooding of the Mississippi River in the summer of 1993. Some eighth-grade students may 
have pets for which they have considered building a house. With regard to the Theme block 
on the flooding in the summer of 1993, most eighth-grade students had just completed their 
fifth-grade year at that time and may have been exposed to media attention about the floods. 
The flooding of the Mississippi River also was the context of the unreleased Theme block 
for grade 12. Seven of the 11 questions in that block were given to both eighth- and 
twelfth-grade students. 

Students taking the Doghouse block of questions were given a sheet of push-outs 
representing parts of a doghouse. A picture of the sheet follows. The sheet included a set of 
seven push-outs that were parts of the doghouse — two roof pieces, two side walls, a front wall, 
a back wall, and a floor; a set of two push-outs (indicated with an “A” and a “B”) that when 
folded together formed a model of the doghouse; and a separate push-out (indicated with a “C”) 
that represented the door opening of the doghouse. Students were also provided with a 
ruler/protractor and a calculator. 
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Both Theme blocks included multiple-choice and constructed-response questions. 

The Doghouse block included four multiple-choice and six constructed-response questions, 
whereas the Flooding-of-the-Mississippi block included four multiple-choice and seven 
constructed-response questions. In two of the questions in the Doghouse block, students were 
asked to show work that supported their answers. In the Flooding block, students were asked to 
explain their answers in five of the questions. 

Overall student performance 

Students’ overall performance on the two blocks is presented in Table 3.12. The average 
percentage correct score was 41 percent for the Doghouse block and 30 percent for the 
Flooding block. In the Doghouse block, female students had a significantly higher percentage 
correct score than male students. White, Hispanic, and Asian/Pacific Islander students 
outperformed Black students, and White and Asian/Pacific Islander students also outperformed 
Hispanic students. For the Flooding block, male and female students performed similarly, 
while the pattern of performance by racial/ethnic groups was the same as noted on the 
Doghouse block. The frequency with which students engaged in the two classroom practices 
highlighted in this chapter was not found to be related to student performance. The sample of 
American Indian students for both Theme blocks was too small to permit reliable estimates of 
their performance on either the blocks as a whole or on individual questions. Therefore, the 
performance of American Indian students is not discussed. 
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©rad© $, 











Build g Doghouse 


Flooding 


© 






All Students 


41 


30 


Gender 






Males 


39 


31 


Females 


43 


30 


Race/Ethnicity 






White 


45 


34 


Black 


27 


18 


Hispanic 


33 


22 


Asian/Pacific Islander 


43 


35 


American Indian 


★ ★ ★ 


* * * 


Students Whose Teachers Report 
Asking Students to Write 
a Few Sentences About How 
to Solve a Mathematics Problem... 






Nearly Every Day 


42 


29 


Once or Twice a Week 


44 


32 


Once or Twice a Month 


40 


30 


Never or Hardly Ever 


42 


31 


Students Whose Teachers Report 
Asking Students to Write Reports 
or Do a Mathematics Project... 






Nearly Every Day 


★ ★ ★ 


★ ★ ★ 


Once or Twice a Week 


39 


30 


Once or Twice a Month 


42 


32 


Never or Hardly Ever 


42 


30 



c 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The introduction to the Doghouse block is shown in Figure 3.2. The instructions clarified what 
was expected of students with regard to responses. The description of the context for the block 
of questions brings the students directly into the task by telling them that Julie would like their 
help in building a doghouse. Students were asked first to put together a model of the doghouse 
using pieces “A” and “B” and following the directions given. The results of these efforts were 
not, however, collected or scored. 
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Sifodk, ©mde 8, 1996 



THE NATION’S 




This part has 10 questions. Mark your answers in your booklet. You will 
have to fill in an oval or write your answer as directed. In those questions 
where you must write an answer, it is important that your answer be clear 
and complete and that you show all of your work since partial credit may 
be awarded. The last two questions may each require 5 minutes or more to 
think about and answer. After each question, fill in the oval to indicate 
whether you used the calculator. If you are asked to round your answer, do 
not round any numbers except your final answer. 




Julie wants to build a doghouse like the one shown in the picture above. 
She has asked you to help her build the doghouse. 

The kit you have been given contains a model of a doghouse like the one 
Julie wants to build. Please put the model together now by following these 
instructions. 

1 . Separate pieces A and B from the paper. Do not separate any other 
pieces from this paper until you are told to do so. 

2. Fold up the four walls on piece A so that they form right angles with 
the rectangular floor. 

3. Fold the roof (piece B) in half, and set it on top of the house. The 
edges of the roof will extend slightly beyond the walls. 

Note: When Julie builds the house, the roof will be made up of two 
identical pieces of wood, since wood cannot be folded the way you 
folded the piece of paper just now to make the roof. 

You may also use your calculator and ruler/protractor to help answer the 
questions in this part. 
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Question 1. Identifying needed information. The first actual question the students 
encountered asked them to consider whether each of five different measurements would help 
Julie determine whether the finished doghouse will be large enough to accommodate her dog. 
The question was designed to assess content from the Geometry and Spatial Sense content 
strand and the mathematical ability Conceptual Understanding. Students were asked to reply 
“yes” or “no” to the utility of each of the measurements, and each response was scored 
“correct” or “incorrect.” Only measurement “d” contained specific mathematical language 
(i.e., “rectangular”). 



L Consider each of the following measurements. Will knowing the 
measurement help Julie to determine whether the doghouse she plans 
to build will be large enough for her dog to sleep in and to go in and 
out of comfortably? (Answer "Yes" or "No" for each part.) 



a. 


The length of the floor 


O 


Yes 


O No 


b. 


The height of the house 


O 


Yes 


O No 


c. 


The weight of the house 


o 


Yes 


O No 


d. 


The width of the rectangular floor 


o 


Yes 


O No 


e. 


The width and height of the door's opening 


o 


Yes 


O No 



Student performance information for this question is presented in Tables 3.13 and 3.14. 
The data in Table 3.13 summarizes the percentage of students by the number of measurements 
correctly identified as useful in helping Julie determine whether the doghouse would 
accommodate her dog; Table 3.14 details the percentage correct scores for individual 
measurements. Students appear to have done relatively well on this question. Fifty-five percent 
of the students correctly evaluated the utility of each of the five measurements, and 23 percent 
evaluated four of the five measurements correctly. 
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THE NATION’S 


TMkO© 


Score Percentages for 


REPORT 

CARD 


rains 


"identifying Needled information/' Grade 8 









5 Correct 


4 Correct 


3 Correct 


2 Correct 


1 Correct 


0 Correct 


Omit 


















All Students 


55 


23 


9 


6 


6 


i 


0 


Males 


51 


24 


10 


6 


7 


i 


0 


Females 


60 


22 


7 


6 


4 


0 


0 


White 


62 


20 


7 


5 


5 


1 


0! 


Black 


34 


30 


14 


10 


10 


1 


1 


Hispanic 


35 


37 


13 


7 


6 


1! 


0 


Asian/Pacific Islander 


73 


16 


5! 


4 


1! 


1! 


0! 


American Indian 


* * * 


★ ★ * 


* ★ ★ 


★ ★ ★ 


* * * 


* * * 


* * * 


. 








.. .. 


... * 







NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

***$ample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 




Percentages Correct 
'Identifying Needed Information, 



= THE NATION’S 
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m 





la - Yes 


lb — Yes 


ic - m 


Id - Yes 


le - Yes 














All Students 


90 


86 


72 


82 


83 


Males 


87 


85 


69 


79 


81 


Females 


92 


87 


76 


86 


85 


White 


91 


88 


77 


85 


86 


Black 


82 


77 


58 


70 


74 


Hispanic 


86 


83 


61 


79 


74 


Asian/Pacific Islander 


98! 


94 


84 


86 


90 


American Indian 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 



***Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The data in Table 3.14 show that, in general, students appear quite knowledgeable about 
the use of each of these measurements: that is, for each measurement more than 70 percent of 
students were able to assess correctly whether the measurement would help Julie. Students had 
more difficulty correctly assessing what appeared to be more complicated or more formal 
measurements. For example, they were better at assessing correctly whether the “length of the 
floor” or the “height of the house” would be helpful than they were at assessing the usefulness 
of the “width of the rectangular floor” or the “width and height of the door’s opening.” 
Interestingly, students had most difficulty determining whether the “weight of the house” would 
be a helpful measurement, or they may have been inclined to answer “yes” to measurement “c” 
because “yes” was the correct answer to each of the other measurements presented. 
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Quiestioini 2. Determine mmiDniiim measiiriirng needed. In the second question, students were 
first given information about the pieces that made up the doghouse — four walls, two roof 
pieces, and the floor. They were then told that some of the pieces were exactly the same size 
and shape, and that Julie did not want to measure all of the pieces, if not necessary. Students 
were presented with the problem of determining the smallest number of individual pieces Julie 
would need to actually measure in order to have the information she needed to cut out all of the 
pieces for the doghouse. The question was classified as a Geometry and Spatial Sense question 
and was designed to assess Conceptual Understanding. 

In order to answer this question correctly, students could use either the model of the 
doghouse or the push-out pieces to determine that there were three matched pairs among the 
seven pieces that Julie needed to cut; therefore, she only needed to make four unique 
measurements. The problem was simplified if students realized that they did not need to 
actually measure any of the pieces themselves. 



2. Seven pieces — four walls, two roof pieces, and the floor — make up 
the doghouse. Since some of the pieces are exactly the same in size 
and shape, Julie does not need to measure every piece. She can 
measure and cut a piece and then make identical pieces without 
measuring by tracing an outline of the cut piece onto the wood and 
then cutting out the traced shape. 

How many of the seven pieces does Julie need to measure before she 



cuts? 




<£> 


Two 


d> 


Three 


O 


Four 


CD 


Five 


CD 


Seven 
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Information on student performance on this multiple-choice question is presented in 
Table 3.15. Forty-two percent of the students were able to correctly identify four pieces as the 
correct response. The second highest percentage, 24 percent, chose Option B, three pieces. 
There are a number of reasons that students could have chosen three pieces. This is the answer 
students would arrive at if they assumed that all the walls were the same size, the two roof 
pieces were of the same size, and the floor was of a size different from the walls or roof pieces. 
This would also be the correct response if students assumed that they could use the two 
adjacent walls to trace out the floor and therefore did not need to measure the floor piece itself. 






Percentage Correct 






All Students 


42 


Males 


40 


Females 


43 


White 


46 


Black 


28 


Hispanic 


32 


Asian/Pacific Islander 


37 


American Indian 
i 


* * * 



***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 

Question 3. Measure lengths using ruler. The third question asked students to actually 
measure, in inches, the pieces of the model doghouse that they put together initially. For 
example, students were told to measure “the longer side of the rectangular floor.” This meant 
that the student had to know what a “rectangular floor” was, determine which was the 
“longer side,” and use the ruler correctly to measure. This question was designed to assess 
Measurement content and Procedural Knowledge. 



3. The model of the doghouse that you put together is a smaller version 
of the actual house. Measure the following lengths, in inches, on your 
model and record your results in the spaces below. 

Longer side of rectangular floor inches 

Shorter side of rectangular floor inches 

Height from floor to highest point 
of roof inches 
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The responses to this question were scored on a 3-point rubric: “complete,” “partial,” 
and “incorrect.” As shown below, a “complete” response gave three correct measurements: two 
inches for the longer side of the rectangular floor, one-and-one-half inches for the shorter side of 
the rectangular floor, and two inches for the height from the floor to the highest point of the roof. 

SaamjpUxB 66 (smmfpE(BU(B 99 wcBsjpciDmsie 



3. The model of the doghouse that you put together is a smaller version 
of the actual house. Measure the following lengths, in inches, on your 
model and record your results in the spaces below. 

Longer side of rectangular floor inches 

Shorter side of rectangular floor 1 >*3 inches 

Height from floor to highest point 
of roof inches 



A partial” response provided only one or two correct measurements in inches. In the 
sample partial” response shown, it appears that the student may have been careless in 
conducting the measurements and used the centimeter side of the ruler to measure the longer 
side of the rectangular floor, arriving at 5 (centimeters rather than inches). The remaining 
dimensions were measured correctly in inches. 

ScrofflMjpfl® 66 jpaad°itu(!aU 99 d°(BSfp(wms(B 



3. The model of the doghouse that you put together is a smaller version 
of the actual house. Measure the following lengths, in inches, on your 
model and record your results in the spaces below. 

Longer side of rectangular floor 

Shorter side of rectangular floor 

Height from floor to highest point 
of roof 



inches 



% inches 

inches 
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Finally, an “incorrect” response gave no correct measurements in inches. The 
measurements in the sample “incorrect” response appear to have been measured in centimeters 
rather than inches. It is not clear if the student did not understand the difference between 
centimeters and inches or if the student simply was careless and used the wrong side of the 
ruler to conduct the measurements. 

Sumple tnr<B<sit 99 r®sj demise 



3. The model of the doghouse that you put together is a smaller version 
of the actual house. Measure the following lengths, in inches, on your 
model and record your results in the spaces below. 

Longer side of rectangular floor 5 inches 

Shorter side of rectangular floor U inches 

Height from floor to highest point p* 
of roof J inches 



Student performance information on this question is presented in Table 3.16. The modal 
score for the responses was “complete” (46%) with similar percentages of responses scored 
“partial” and “incorrect” (23% and 22%, respectively). 




Score Percentages for 
y Measure Lengths Using Ruler," Grade 8 



= THE NATION'S 
REPORT tosrai 





Complete 


Partial 


Incorrect 


Omit 


®x°x°te ©;V.y 










All Students 


46 


23 


22 


8 


Males 


46 


22 


23 


8 


Females 


47 


24 


21 


8 


White 


54 


21 


18 


6 


Black 


17 


31 


34 


17 


Hispanic 


36 


25 


31 


8 


Asian/Pacific Islander 


48 


24 


25 


2! 


American Indian 


* * * 


* * * 


* * * 


* * * 



NOTE: Row percentages may not total 1 00 due to responses rated “Off Task” or to rounding, or both. 

***Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 4. Apply concept of ratio. The fourth question was prefaced by a scale that 
showed an inch representing IV 2 feet (18 inches). Students were asked to explain how they would 
use the scale to determine in feet the measurements they obtained in inches in Question 3. 
Students were also told that they were not required to find the actual measurements; that is, 
students were simply being asked to provide an explanation of the procedure they would follow 
to convert measurements in inches to measurements in feet when given the conversion scale. 
This question was classified as a Number Sense, Properties, and Operations question and was 
designed to assess Conceptual Understanding, specifically, of the use of ratios. 



1 inch represents 1 5 feet (18 inches) 



4 . Explain what you would need to do to each of your measurements in 
question 3 to find the measurements in feet of the actual house. 

(You do not have to find each of the actual measurements.) 



Responses to the question were scored using a 3-point rubric: “complete,” “partial,” 
and “incorrect.” A “complete” response provided an explanation of the correct procedure for 
translating the inches into feet using the scale provided. The sample response shown below 
describes that procedure in one sentence. Although not necessary, it also provides the 
measurements in feet. 

Sosfisopte 66 <i?©0sb2b0©#<b 99 
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Responses were scored as “partial” if they met any of the following criteria: 

O the correct procedure for finding the measurements in inches rather than feet; 

O the correct procedure using one of the measurements from Question 3, but not 
generalizing to the other measurements; 

O the three correct measurements in feet, but no explanation of the procedure; or 

O an example of the correct procedure using one of the measurements in Question 3 
multiplied by 18, 1.5, or 18/12. 

The “partial” response that follows provides a procedure for finding the measurement in inches 
rather than feet. 

Sample ** partial 99 response 

\ \V\ ^ \J)<?* you 

w tU'/t 1 ii'tci'i fo /y /licit/, 

An “incorrect” response met none of the previous criteria. The following “incorrect” 
response begins with an attempt to explain how to solve the problem, “To find the actual 
measurements...,” and ends the sentence with the general procedure to convert inches into feet 
with no reference to use of the scale provided. 

Sample “ incorrect 99 response 

*<3d thx- QxdtuuoJ ip% iO&iilldl hu&'fe doiMM/g 

'nwrnl>tf?6 Jby /d Jbecfa&dL 4hms a 

/Pff mnoSid i^uid v •: • *•;' 

Student performance data for this question are presented in Table 3.17. Fourteen 
percent of the students were able to provide “complete” responses; however, 45 percent of 
students were not able to provide even a partially correct answer. It is not clear whether 
eighth-grade students do not understand how to use a scale, or whether they cannot explain in 
words a general mathematical procedure, or both. Nevertheless, there is certainly hesitancy on 
the part of many students to attempt to respond to a question like this because a fifth of the 
students chose not to attempt to answer this question at all. 




30 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



THE NATION'S 






Complete 


Partial 


Incorrect 


Omit 












All Students 


14 


17 


45 


21 


Males 


14 


16 


44 


24 


Females 


15 


19 


46 


18 


White 


17 


21 


44 


16 


Black 


3 


6 


53 


36 


Hispanic 


8 


10 


49 


31 


Asian/Pacific Islander 


22 


21 


40 


16 


American Indian 


★ ★ * 


★ ★ ★ 


* * * 


* * * 


1 



NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 

***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 

Question 5. Understand concept of ratio (I). The fifth question students encountered was a 
multiple-choice question classified in the Number Sense, Properties, and Operations content 
strand that was designed to assess Conceptual Understanding, specifically of the use of scales 
and ratios. The question asked students to select, from among five options, the scale that would 
produce the largest doghouse. There are a number of different strategies students could use to 
solve the problem and thereby select the correct option. One strategy, for example, would 
involve referencing all of the scales to the same number of inches; that is, convert all of the 
scales so that each indicates the number of feet represented by one inch. The scale with the 
largest number of feet is then the correct response. Some students also could have been able 
immediately to see ways to eliminate certain options before doing any computations. 

For example, some students may have seen that Option D would produce a larger doghouse 
than Option E, and that Option B would produce a larger doghouse than Option C, and 
therefore eliminated Options E and C at the outset. 



5. Of the following scales, which one would produce the largest 
doghouse? 

CD 2 inches on model represents 5 feet on actual house. 

CD 1 inch on model represents 3 feet on actual house. 

CD 1 inch on model represents I2 feet on actual house. 

CD 2 inch on model represents 1 foot on actual house. 

i 1 , 

CD 2 inch on model represents 4 foot on actual house. 

m. 
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Student performance data on this question are presented in Table 3.18. Just over a 
third of the students were able to select the correct scale that would produce the largest 
doghouse, Option B. Forty-seven percent of students selected Option A. From this high 
percentage, it appears that many students did not know how to use scales, and consequently, 
used other reasoning in selecting their answer. Option A has both the largest number of inches 
and the largest number of feet, so it appears that many students may have simply selected that 
option because it had the largest numbers. It is not clear, however, from students’ choices, 
whether they have little or no understanding of scales or simply do not know how to compare 
differing ratios. 
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'Understand Concept of Ratio (!)/ 
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Percentage Correct 
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All Students 


35 


Males 


39 


Females 


31 


White 


38 


Black 


21 


Hispanic 


27 


Asian/Pacific Islander 


42 


American Indian 


* * * 


r 



***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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Question. 6. Understand concept of ratio (II). In the sixth question, students were provided 
with a scale they were to use in answering the question and were asked to indicate, based on 
the scale provided, how much taller the actual doghouse would be compared to the model. 

This question also was designed to assess Conceptual Understanding, specifically the use of 
scales and ratios, and content from the area of Number Sense, Properties, and Operations. 



1 inch represents l^feet (18 inches) 



6. The height of the actual house will be how many times as tall as the 
height of the model? 



CD 


1 1 

1 2 


CD 


9 


CD 


18 


CD 


24 


CD 


27 



If, in answering the question, studentjs understood the use of scales and also realized 
that they should be comparing inches to inches rather than converting the height of the actual 
house to feet, the correct response, Option C, would be straightforward. That is, they would see 
that an inch on the model represents 18 inches of the actual doghouse, and that therefore the 
actual doghouse would be 18 times as tall as the height of the model. Student performance data 
on this question are presented in Table 3.19. Thirty-five percent of the students were able to get 
the correct answer. The next highest percentage of students, 26 percent, selected Option A, 
which specified that the actual doghouse would be one-and-a-half times as tall as the model. It 
may be that this option attracted students who did not think about the metric in which the height 
was being measured, or they may have been drawn to it because of the repeated number (1V6). 
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All Students 


35 


Males 


34 


Females 


35 


White 


38 


Black 


24 


Hispanic 


30 


Asian/Pacific Islander 


39 


American Indian 


★ ★ ★ 


L 



***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



Question 7, Correctly position doon The seventh question required students to trace the 
correct position of the door opening onto a drawing of the front wall of the doghouse, using 
push-out piece “C” as a template. Students were told how to position the door, using the 
measurements of the actual doghouse, and given a conversion scale for the model pieces. In 
addition, they were cautioned that this conversion scale was not the same as the one used in the 
previous question. This question was classified as a Geometry and Spatial Sense question and 
was designed to assess Problem-Solving ability. 
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7. You will mow meed piece C to answer this question. Separate 
the paper. 



piece C from 



2 inches represents 1 foot (12 inches) 



Piece C represents a scale model of the door for the doghouse. The front wall of the 
doghouse, shown below, as well as piece C has been drawn to a different scale than the 
one used in the previous question. The scale is 2 inches represents 1 foot. 

On the drawing below, use piece C to locate the door on the wall so that it will be ^ foot 
above the floor level of the doghouse (to keep the water out) and centered exactly 
between the vertical edges of the wall. When you have correctly positioned the door, 
trace its location on the drawing. (Disregard the thickness of the wood that will be used 
to build the doghouse.) 




Floor 



TT 
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In order to solve this problem correctly, students had to convert half-a-foot above the 
floor level to one inch above the floor line on the drawing of the front wall. There were several 
strategies students could use to center the door opening. One of the simplest is to measure both 
the width of the front wall and the width of the door opening, find the centers of both, and align 
the centers. Finally, the student had to be able to trace the door opening in the correct position. 

The responses to this question were scored on a 3-point rubric: “complete,” “partial,” 
and “incorrect.” “Complete” responses had the bottom of the door opening positioned between 
and including 0.906 to 1.064 inches above the floor and the sides of the door opening 
positioned between and including 1,142 to 1.314 inches from the side edges of the front wall. 

A “complete” response follows. 

!§mmipU@ r<B§pmm§<& 
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A “partial” response either had the door centered correctly but not located an inch 
above floor level, or located an inch above the floor level but not centered correctly. The door 
opening in the “partial” response shown below is correctly centered but only half an inch above 
the floor of the doghouse. Although the exact reason for this mistake is not clear, the student 
appears to have forgotten about the scale and instead erroneously translated the half-a-foot 
instruction into half an inch on the drawing. 
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Responses scored as “incorrect” met none of the criteria mentioned above. In the 
following “incorrect” response, the door opening appears to be a free-hand drawing that is larger 
than the push-out, less than an inch above the floor, and not centered properly. If the drawing 
shown was actually traced, as required by the question, the tracing was rather imprecise. 

SmmjpUcB 66 nim?(E)V o/ Q o (B(Etl 99 




98 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



Information on how students performed on this question is presented in Table 3.20. 
Almost a fifth of the responses were scored as “complete,” and 44 percent were scored 
as “partial.” 
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Complete 


Partial 


Incorrect 


Omit 












All Students 


19 


44 


26 


10 


Males 


16 


42 


29 


13 


Females 


21 


46 


24 


8 


White 


23 


45 


25 


7 


Black 


5 


40 


31 


24 


Hispanic 


1 1 


43 


30 


16 


Asian/Pacific Islander 


14 


40 


32 


15 


American Indian 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 


i 



NOTE: Row percentages may not total 100 due to responses rated "Oft Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 8. Visualize cnt-onts on grid. The eighth question that students encountered was 
classified as a Problem-Solving question in the Measurement content strand. Students were 
instructed to use the seven pieces of the doghouse — the four walls, two roof pieces, and the 
floor — to help answer the question. They were presented with a scale, told that the plywood 
sheets from which doghouse pieces would be cut were four feet wide by eight feet long, and 
provided with drawings of three plywood sheets. (Only one piece is shown here.) The students 
were further told that the pieces of the doghouse and the representations of the plywood sheet 
were all drawn to the same scale. Students were asked to trace the doghouse pieces onto the 
drawings of the plywood sheet in order to demonstrate the fewest number of plywood sheets 
needed to cut out all seven pieces of the doghouse. 



8. Separate the remaining seven pieces from the paoen a 
those seven pieces to help von answer the following qi 


Not shown to size 

,nd use only 
nestiom. 




1 inch represents \\ feet (18 inches) 






Julie plans to use plywood to build her doghouse, using the scale 
above. The plywood is sold in rectangular sheets that are each 4 feet 
wide and 8 feet long. She wants to determine the fewest number of 
sheets that she will need. 

On the grids below and on the next page, the plywood sheets have 
been drawn to the same scale as the seven pieces. Show how the 
seven pieces (four walls, two roof pieces, and the floor) could be cut 
from the plvwood sheets so that the fewest number of sheets are 
used. This should be done by tracing the pieces on the sheets. 

(Note: There may be more sheets shown than you will need to use.) 
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The responses to this question were scored with a 3-point rubric: “complete,” “partial,” 
and “incorrect.” The fewest number of plywood sheets needed for the seven pieces was two. A 
“complete” response was one in which the seven pieces were correctly drawn onto two plywood 
sheets. Responses that showed two side pieces, two front or back wall pieces, or two roof pieces 
drawn as one piece but correctly labeled were considered correctly drawn. Although students 
were not required to label the pieces, if they did so incorrectly, their responses were not scored 
as “complete.” The following “complete” response is just one of several configurations that 
would accommodate the seven pieces on two plywood sheets. 

SaamjpUcB 66 (£dDmpI}<BtkB 99 
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Responses meeting one of the following criteria were scored as “partial”: 
o six of the seven pieces correctly traced on two sheets with the seventh piece missing; 

o six of the seven pieces correctly traced on two sheets with the seventh piece drawn 
incorrectly, but the seventh piece is not the door opening (piece “C”) or a duplicate 
piece already accounted for; or 

o seven pieces correctly traced on to two sheets, but also piece “C” drawn on one of the 
two sheets or on a third sheet. 



102 



o 



Estimation Skills, Malhemalics-in-Context, and Advanced Skills in Mathematics 



If pieces were labeled incorrectly, responses were still scored as “partial.” In the 
“partial” response shown below, the student appears to have drawn one of the side walls larger 
than the push-out piece provided. 

S mmpUcB v°<B8jp<tt>ms<B 
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An “incorrect” response is one that used three sheets, had drawings of pieces that 
appeared to be freehand or not drawn to scale, or included more than one of any of the seven 
requisite pieces. In the “incorrect” response shown below, it is apparent that although the roof 
pieces, front wall, and back wall were traced, the side walls and floor were drawn freehand and 
were not of the correct size. 
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As the data in Table 3.21 show, 48 percent of the students understood and correctly 
solved the problem. This was a problem where few students provided responses that were 
scored as “partial.” That is, if students were able to correctly trace and position six of the seven 
pieces, it was uncommon for them not to succeed with the seventh piece as well. A substantial 
percentage of the responses, 36 percent, were scored as “incorrect.” In addition, 10 percent of 
the students chose not to attempt this question. 
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Complete 


Partial 


Incorrect 


* 

Omit 


©waste © 










All Students 


48 


7 


36 


10 


Males 


43 


8 


37 


12 


Females 


54 


6 


34 


7 


White 


56 


7 


30 


7 


Black 


20 


2 


58 


20 


Hispanic 


36 


6 


45 


13 


Asian/Pacific Islander 


45 


1 1 


34 


10 


American Indian 


* * * 


★ ★ * 


* ★ ★ 


* * * 
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***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 9. Apply geometry in model. The ninth question was a multistep question classified 
as Problem-Solving and designed to assess content from both the Algebra and Functions and 
the Geometry and Spatial Sense content strands. In the introduction to the problem, students 
were told that they were required to show how they solved the problem and to explain their 
reasoning process. A scale that students had seen in previous problems was provided. The 
question began with a definition of the pitch of a roof. Students were told that Julie had read 
that the optimal pitch for best air flow was 30°, and that the pitch of the model doghouse was 
53°. They then were asked to determine the number of feet by which the height of the actual 
doghouse would decrease if Julie decreased the pitch from 53° to 30°. 



This question requires you to show your work and explain your 
reasoning. You may use drawings, words, and numbers in your 
explanation. Your answer should be clear enough so that another person 
could read it and understand your thinking. It is important that you show 
all of your work. 



1 inch represents ljfeet (18 inches) 



9. The drawing below shows a wall of Julie's doghouse. The pitch is 
defined as the slope of the roof; it can also be described as the angle 
formed between the roof and a horizontal line, as shown in the 
drawing. 




The pitch of the roof in the drawing is slightly more than 53 degrees, 
which is the same as the roof pitch on your model. 

Julie read in a book that the best air flow inside a doghouse occurs 
when the roof pitch is 30°. If the height of the doghouse is measured 
from the floor to the highest point on the roof, by about how many 
feet is the actual height of the doghouse decreased when the pitch is 
decreased from 53° to 30°? 

Show how you got your answer. (You may find it helpful to mark on 
the drawing.) 
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To solve the problem, students needed to understand the definition of pitch and be able 
to draw a new roof slope based on a pitch of a different degree. That is, they needed to know 
what an angle is, what a horizontal line is, and how to use a protractor to measure an angle. 
Finally, students needed to know how to use the scale provided to convert measures on the 
drawing of the model to measures of the actual doghouse. Responses to this question were 
scored using a 4-point rubric: “satisfactory,” “partial,” “minimal,” and “incorrect.” The 
responses were scored on the presence of three attributes: 

1. correct indication of the 30° angle on the drawing; 

2. correct measurement of the height of the old doghouse, the height of the new 
doghouse, or the sides opposite the 30° angle in the right triangle; or correct 
subtraction or measurement to obtain the difference between the heights of the 
old and new doghouses; and 

3. correct conversion of the decrease in height from inches to feet using the scale 
provided. 

A “satisfactory” response included all three of the attributes detailed above. The 
sample “satisfactory” response shown has the correct indication of the 30° angle on the 
drawing, the correct measurement (with explanation) of the decrease in the height of the roof 
from the old to the new, and proper conversion of inches to feet using the scale provided, 
resulting in the correct answer of % of a foot. 
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A “partial” response had two of the attributes detailed above. The following “partial” 
response has the 30° angle correctly measured and drawn and has the correct measurements of 
the old and new heights for the full-scale doghouse shown in inches. The response also 
includes the decrease in the height but leaves it in inches and does not carry out the conversion 
to feet. 

SSmmjjDlte d°<B§p(iDM§<& 
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A “minimal” response has one of the correct attributes. The sample “minimal” response 
shows a correct measure of the new height of the model doghouse. It also includes the decrease 
in heights of the old and new models but does not show or mention the use of the 30 ° angle and 
does not convert inches into feet using the scale provided. 




An “incorrect” response had none of the correct attributes. The sample “incorrect” 
response shows little understanding of how to approach solving the problem. 

$§<mmpl@ response 
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The data in Table 3.22 show that nearly half of the responses from students were 
scored as “incorrect.” Furthermore, only 22 percent of the responses received any credit, and 
24 percent of the students chose not to attempt to solve this problem at all. 
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All Students 


2 


6 


14 


49 
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Males 


1 


6 


13 


47 


28 


Females 


2 


7 


14 


52 


19 


White 


2 


8 


16 


47 


21 


Black 


0! 


0! 


6 


60 


32 


Hispanic 


0! 


4 


8 


53 


32 


Asian/Pacific Islander 


* * ★ 


* ★ ★ 


* * * 


★ ★ ★ 


* ★ ★ 


American Indian 


★ ★ ★ 


★ ★ ★ 


* * * 


★ ★ ★ 


★ ★ ★ 


r 



NOTE: Raw percentages may nat total 100 due ta responses rated "Oft Task" ar ta rounding, ar bath. 

*** Sample size is insufficient ta permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may nat be accurately 
determined and/ar the sampling distribution af the statistics daes nat match statistical test assumptions (see Appendix A). 
SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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Question 10. Find maximum area when perimeter is fixed. The final question was classified 
as a Measurement question and was designed to assess Problem-Solving ability. Students were 
introduced to this question with information about what was required of their responses. Then 
they were presented with the problem. They were told that Julie wanted to build a fence around 
a section of the yard for her dog and that she was able to purchase 36 feet of fencing. They also 
were told what the enclosed area should look like (i.e., a rectangle with whole number lengths). 
Students were asked to determine the largest area that Julie could enclose with her fencing and 
to show work that would convince Julie that their solution had indeed yielded the largest area. 



This question requires you to show your work and explain your 
reasoning. You may use drawings, words, and numbers in your 
explanation. Your answer should be clear enough so that another person 
could read it and understand your thinking. It is important that you show 
aU of your work. 



10. Julie wants to fence in an area in her yard for her dog. After paying for 
the materials to build her doghouse, she can afford to buy only 36 feet 
of fencing. 

She is considering various different shapes for the enclosed area. 
However, she wants all of her shapes to have 4 sides that are whole 
number lengths and contain 4 right angles. All 4 sides are to have 
fencing. 

What is the largest area that Julie can enclose with 36 feet of fencing? 

Support your answer by showing work that would convince Julie that 
your area is the largest. 



To answer this question students needed to know what a rectangle is; have a conceptual 
understanding of the fact that rectangles with the same perimeter could have different areas; 
and be able to calculate the area from the length and width of the sides of the rectangle. This 
question was scored on a 5-point scale: “extended,” “satisfactory,” “partial,” “minimal,” and 
“incorrect.” An “extended” response needed to provide the correct answer of 81 square feet 
along with evidence that all length/width combinations had been considered in determining 
the largest area enclosed. The sample “extended” response clearly explained that the largest 
rectangular area of perimeter 36 feet would be a 9-foot square, which provides an area of 
81 square feet. It also included a table that listed all the possible combinations of widths and 
lengths that would result in a perimeter of 36 feet, thus providing evidence that could 
“convince Julie” and also showing that the student clearly understood the relationships between 
the lengths of the sides of the rectangle, perimeter, and area. 
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“Satisfactory” responses indicated that the 9-by-9 square has the maximum area of 
81 square feet; indicated that another rectangle has the maximum area, due to a mathematical 
error in the accompanying work; or contained calculations for all nine rectangles (i.e., widths of 
1 though 9) but did not specify the maximum area. The sample “satisfactory” response shown 
correctly designates the rectangle with largest area; however there is a computational error in 
the supporting work. 
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A “partial” response was one that a) showed at least three different rectangles with their 
dimensions and areas and might have included an indication that one of those rectangles yielded 
the maximum area, or b) indicated that the 9-by-9 square had the maximum area of 81 square 
feet, but showed no accompanying work. The “partial” response shown below specifies the 
correct rectangle and area (although the area units are incorrectly labeled as “feet” rather than 
“square feet”), but the work shows no consideration of other possible rectangles. 

Sample partial 59 response 
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A “minimal” response must have included some evidence of understanding that area 
and perimeter formulas for rectangles were needed to solve the problem and might have 
included an attempt to organize the data. The sample “minimal” response shows that the 
student had an understanding of using the 36-foot-long fencing to form a rectangle, but was 
unclear as to the exact formula for calculating the area of that rectangle. 

SaamjpUcB 66 mnnwnmmU 99 




An “incorrect” response had none of the preceding criteria. Although it is not definitive 
from the “incorrect” response shown, it appears that the student knew how to calculate the area 
of a rectangle, but did not fully understand the task explained in the problem, since the 
perimeter of the student’s rectangle does not equal 36 feet. 
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Student performance data are presented in Table 3.23. Less than one-half of one 
percent of the responses were scored “extended” or “satisfactory.” A third of the responses 
received at least minimal credit, while 38 percent were scored as “incorrect.” A large 
percentage of students, 27 percent, chose not to attempt this question. * 
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* * * 
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★ ★ * 


★ * * 


★ ★ ★ 


★ ★ ★ 


* * * 


* * * 
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NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Twelfth-grade students who participated in the Theme Study were similar demographically to 
students who participated in the national NAEP assessment in mathematics. The data in 
Table 3.24 show that similar percentages of male and female students participated in each 
of the samples; the majority of students were White students; and the modal response from 
students regarding their parents’ highest level of education was “graduated from college.” In 
addition, the large majority of twelfth-grade students were from public schools, very few 
students were part of the Title I program, and about 10 percent of students were eligible for the 
federal Free/Reduced-Price Lunch program. 

Teachers of twelfth-grade students were not surveyed either in the main NAEP 
assessment or in the Theme Study. Therefore, the information presented on classroom practices 
in Table 3.25 is based on responses from the twelfth-grade students themselves. Students in 
each of the different samples indicated similar levels of frequency of writing about solving a 
mathematics problem and of writing reports or doing a mathematics project. Notice that only 
small percentages of students reported engaging in either of these practices with any frequency, 
while 60-70 percent of students reported “never or hardly ever” doing so. In interpreting these 
data, however, one must keep in mind that the “never or hardly ever” category also includes 
responses from students not currently enrolled in mathematics. In the main NAEP assessment, 
only 62 percent of grade 12 students indicated being enrolled in mathematics. 
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fflain Assessment 


Theme Slock 1 
Buying a Car 


Theme Slock 2 
Flooding 


@otsc°te U2 

Gender 








Males 


48 


50 


49 


Females 


52 


50 


51 


Race/Ethnicity 






69 


White 


70 


69 


Black 


14 


14 


14 


Hispanic 


1 1 


12 


12 


Asian/Pacific Islander 


4 


4 


4 


American Indian 


1 


0 


0 


Students who Reported Their 
Parents' Highest Level of 
Education as... 








Did Not Finish High School 


6 


7 


7 


Graduated From High School 


19 


18 


20 


Some Education After High School 


25 


26 


27 


Graduated From College 


47 


46 


43 


1 Don't Know 


3 


3 


3 


Students who Attend... 








Public Schools 


88 


89 


88 


Nonpublic Schools 


12 


1 1 


12 


Title 1 Participation... 

Participated 


2 


2 


2 


Did Not Participate 


98 


98 


98 


Free/Reduced-Price Lunch 
Program Eligibility... 






10 


Eligible 


13 


10 


Not Eligible 


60 


64 


65 


Information Not Availabile 


27 


26 


25 



I 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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(Vitsin Assessment 


Buying a Car 


Flooding 


'02 








Students who Report Writing 
a Few Sentences About How to 
Solve a Mathematics Problem... 








Nearly Every Day 


6 


6 


! 7 


Once or Twice a Week 


12 


13 


13 


Once or Twice a Month 


18 


19 


18 


Never or Hardly Ever 


64 


63 


63 


Students who Report Writing 








Reports or Doing a 
Mathematics Project... 








Nearly Every Day 


2 


1 


2 


Once or Twice a Week 


4 


4 


5 


Once or Twice a Month 


24 


22 


25 


Never or Hardly Ever 


70 


72 


69 



* Teachers of twelfth-grade students were not surveyed in these assessments. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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As with grades 4 and 8, the two blocks of Theme Study questions at the twelfth-grade level also 
were structured around engaging contexts. The released block involved buying or leasing a car, 
a topic that is likely to be of interest to twelfth-grade students. The unreleased block had the 
same context and many of the same questions as the unreleased block at the eighth-grade level: 

It concerned the flooding of the Mississippi River during the summer of 1993. 

Both of the Theme blocks at grade 12 included multiple-choice and constructed-response 
questions. In the Buying-a-Car block, 2 of the 7 questions were multiple-choice. Four of the 
remaining five constructed-response questions required the students to show how they solved 
the problem. In the Flooding block, 4 of the 11 questions were multiple-choice, and in 4 of the 
7 constructed-response questions, students were asked to show how they arrived at their answers. 

( TDtD<Bv°mM jp<Brr{l(iDv^Mmw(E<B 

Information on students’ performance on the two Theme blocks is presented in Table 3.26. 

The average percentage correct score is 41 percent for the Buying-a-Car block and 38 percent 
for the Flooding block. On both of the Theme blocks, male and female students performed 
similarly. However, on the Buying-a-Car block, White, Hispanic, and Asian/Pacific Islander 
students outperformed Black students, and White students outperformed Hispanic students, 
whereas on the Flooding block, White students outperformed Black, Hispanic, and Asian/Pacific 
Islander students, and Asian/Pacific Islander students outperformed Black students. The sample 
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of American Indian students for both Theme blocks was too small to permit reliable estimates if 
their performance on either the blocks as a whole or on individual questions. Therefore, the 
performance of American Indian students is not discussed. 

In terms of the highlighted classroom practices, the levels of frequency of writing a few 
sentences and of writing reports or doing a mathematics project were not found to be related to 
student performance on the questions in the Buying-a-Car block. Performance on the Flooding 
block, however, was found to have some relationship to the frequency of those classroom 
practices, although not necessarily in the direction that one might predict. The data show that 
students who “nearly every day” are asked to write a few sentences about how to solve a 
mathematics problem were outperformed by students who indicated a lower frequency, and 
students who indicated writing reports or doing mathematics projects “once or twice a week” 
were outperformed by students who indicated a lower frequency. One can only speculate about 
the reasons for these unexpected relationships. It is possible that they reflect ambiguities in 
the meaning of phrases such as “mathematics projects” or other sources of inaccuracies in the 
self-reported answers to the survey questions. Another possibility is that activities such as 
mathematics projects and writing about how to solve mathematics problems are being used less 
frequently in classes that enroll more high achieving students. Finally, it is possible, that 
contrary to expectations, introduction of these types of teaching practices actually decrease 
mathematics learning, perhaps by taking time away from more effective instructional activities. 
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Average Percentage Correct Scores hy Theme Block, Report 
Grade 12, 1996 CARD 



S 





Buying a Cor 


Flooding 


dtosxiks US 






All Students 


41 


38 


Gender 






Males 


40 


39 


Females 


42 


37 


Race/Ethnicity 






White 


46 


44 


Black 


24 


23 


Hispanic 


30 


26 


Asian/Pacific Islander 


45 


34 


American Indian 


* * * 


* * * 


Students Whose Teachers Ask 
Them to Write a Few 
Sentences About How to Solve 
a Mathematics Problem... 






Nearly Every Day 


33 


31 


Once or Twice a Week 


43 


38 


Once or Twice a Month 


43 


40 


Never or Hardly Ever 


41 


39 


Students Whose Teachers Ask 
Them to Write Reports or 
Do a Mathematics Project... 






Nearly Every Day 


it it * 


* * * 


Once or Twice a Week 


37 


31 


Once or Twice a Month 


42 


40 


Never or Hardly Ever 

i 


42 


39 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The released Theme block at the twelfth-grade level is about buying versus leasing a car. 
Unlike the released Theme blocks in grades 4 and 8, this block was not accompanied by any 
supplemental materials other than a calculator. All of the materials or information needed to 
solve the problems were provided in the questions themselves. The introduction to the block of 
items is shown in Figure 3.3. In this introduction, students were told about the expectation for 
completeness of responses, given information that Question 5 in the block might require more 
time for thinking and answering, told that they should indicate whether they used a calculator 
on each question, and instructed to round only at the final answer when calculating. Then, 
students were introduced to the context for the block of questions: A family is trying to decide 
whether to buy a new car with a loan or to lease the car. Although familiarity with buying or 
leasing a car might have some influence on students’ comfort level with the questions in this 
block, prior knowledge about buying and leasing cars was not essential to solving the problems. 




Introduction to "Buying @ Car" Theme Block , 
Grade 12, 1996 
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This part has 7 questions. Mark your answers in your booklet. You will 
have to fill in an oval or write your answer as directed. In those questions 
where you must write an answer, it is important that your answer be clear 
and complete and that you show all of your work since partial credit may be 
awarded. Question 5 may require 5 minutes or more to think about and 
answer. After each question, fill in the oval to indicate whether you used the 
calculator. If you are asked to round your answer, do not round any numbers 
except your final answer. 

A family is trying to decide how to pay for the new car they want. Should 
they buy the car with a loan or should they lease it? If they lease the car, 
then they must return the car to the dealer at the end of the lease period. If 
they buy the car, then at the end of the loan period the car belongs to them. 
The monthly payments for the car, whether they buy it or lease it, will 
depend upon the size of their down payment. If they make a big down 
payment now, they will have smaller monthly payments, but they might 
like to use the down payment money to buy something else. How will the 
interest rate and the period of time for the loan or lease affect the total 
amount they will pay? 

All the problems on this test are related to financial questions similar to 
those asked above that involve buying or leasing a car. 
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Question 1. Find amount of down payment. In the first question, students were introduced to 
Donna, who wants to buy a car, and were provided with the selling price of the car she wants to 
buy, $16,500. They were asked to determine the required down payment in dollars, given a 
down payment requirement of 20 percent of the selling price. This is a relatively 
straightforward computational question that was designed to assess content from the Number 
Sense, Properties, and Operations content strand and Procedural Knowledge. One way to 
answer the question is to translate the 20 percent into the decimal 0.20 and then to multiply 
the selling price by the decimal fraction. However, students could have used a number of other 
strategies. For example, some students may have known that 20 percent is equivalent to 
1/5 and multiplied the selling price by 1/5 or divided it by 5. Others may simply have used the 
percent key on their calculators. 



1. Donna decides to buy a new car that is selling for $ 1 6,500. If she is 
required to pay 20 percent of the selling price as a down payment, what 
is the number of dollars required for the down payment? 




$330 




$1,650 


<5D 


$3,300 




$4,125 




$13,200 



The correct response to this multiple-choice question is $3,300, Option C. 

Student performance information is presented in Table 3.27. A large majority of the students, 
82 percent, were able to answer this question correctly. 



12 3 
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Percentage Correct 


@0*3303 




All Students 


82 


Males 


82 


Females 


83 


White 


86 


Black 


73 


Hispanic 


70 


Asian/Pacific Islander 


83 


American Indian 


* * * 


I 



***$ample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 

Question 2. Find total amount paid for car. In the introduction to the second question, 
students were told that the information given was to be used in Questions 2 and 3. The 
information included the $19,200 selling price of Bill’s car; his down payment of $4,800; and 
the duration of his loan, 36 months. In Question 2, students were asked to determine how much 
Bill paid in total for his car, given the down payment and monthly payments of $451. This 
question was classified as a Number Sense, Properties, and Operations question and was 
designed to assess Procedural Knowledge. Like the first question, this was a relatively 
straightforward computational question. The one difficulty that students might have had was in 
realizing that the selling price of the car, $19,200, was extraneous to answering the question. 



Questions 2-3 refer to the following information. 

Bill purchased a new car that was selling for a price of $ 19,200. He paid 
$4,800 as a down payment and obtained a 36-month car loan to finance the 
remainder of the selling price. 



2. If his monthly payments were $451, what was the total amount, including 
the down payment, that Bill paid for the car? 



CD 


$11,436 


CD 


$16,236 


CD 


$19,200 


O 


$21,036 


CD 


$24,000 
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The data on students’ performance presented in Table 3.28 show that a large percentage 
(83%) of students were able to select the correct option, D. The second highest percentage of 
students, seven percent, selected Option B, which is the number of dollars for the 36 monthly 
payments of $451 without the down payment. Five percent of the students selected Option E, 
which is the sum of the selling price of the car and the down payment. 
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Percentage Correct 


@cKaib US 




All Students 


83 


Males 


83 


Females 


82 


White 


89 


Black 


62 


Hispanic 


74 


Asian/Pacific Islander 


77 


American Indian 


* * * 


i 



***Sample size is insufficient ta permit a reliable estimate. 

SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 



Question 3. Find difference between total amount paid and price. The third question builds 
on the results of the second question: students were asked to indicate how much more Bill paid 
for his car, that is, with the down payment and the loan, than the selling price of the car. This 
question, like Questions 1 and 2, was designed to assess content from the Number Sense, 
Properties, and Operations strand and Procedural Knowledge. 



3. By how much did the total amount Bill paid for the car exceed the 
selling price for the car? 
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In order to answer this short constructed-response question correctly, students needed 
to determine which numbers they needed for solving the problem and perform the subtraction 
accurately. The question was scored simply “correct” or “incorrect”; that is, students were 
not given partial credit for their work. To be scored “correct,” the response must have been 
$1,836 (that is, $21,036 — $19,200). Any other response was scored as “incorrect.” 
Information on student performance on this dichotomously-scored question is presented in 
Table 3.29; 80 percent of the students were able to determine the correct answer. 
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Percentage Correct 






All Students 


80 


Males 


79 


Females 


80 


White 


86 


Black 


60 


Hispanic 


69 


Asian/Pacific Islander 


75 


American Indian 


* * * 


L 



***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 4. Find amount to be financed. The fourth question introduced the term “residual 
value.” Students were asked to detennine the number of dollars that would be needed to 
finance a two-year lease on a car, if the amount to be financed is the selling price minus the 
sum of the down payment and the residual value of that car at the end of the lease period. 

They were given the down payment, $1,500; the selling price of the car, $18,700; and were told 
that the residual value of that car after a two-year lease was 53 percent of the selling price. 

This question also was classified as a Number Sense, Properties, and Operations question and 
was designed to assess Procedural Knowledge. Students were asked to show their work. 



4. When leasing a new car, the amount to be financed is the amount 
left when the down payment and residual value are subtracted from 
the selling price. (The residual value is the value of the car at the 
end of the lease period.) The residual value for a two-year-old car, 
whose original selling price was $18,700, is estimated to be 53 
percent of the selling price. If the amount of the down payment is 
$1,500, what is the number of dollars to be financed for a two-year 
lease? Show the work that led to your answer. 



In essence, students were asked to use knowledge and skills similar to those that they 
needed for the first three questions, only this time in combination, to solve a multistep problem. 
The computational skills required to solve the problem were relatively simple; the difficult part 
of the problem was in setting up the problem and determining the steps involved. One way to 
solve for the amount to be financed is first to calculate the residual value, 53 percent of 
$18,700 or $9,911; add that amount to the down payment to get $11,411; and, finally, subtract 
that sum from the selling price to get $7,289. 
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The responses to this question were scored on a 3-point rubric: “complete,” “partial,” 
and “incorrect.” A “complete” response had the correct answer and showed an acceptable 
process for getting to that answer. The sample “complete” response shows the process through 
mathematical calculations. 

Smmpim 66 emdBjpU<Bit(B 99 d°<BSjpciDms(B 




The following types of responses were scored as “partial” responses: 
° the correct residual value — $9,911; 



° the correct difference between selling price and residual value — $8,789; 

° an incorrect residual value, but with all calculations performed correctly — for 
example, finds 53 percent of $17,200, which is the selling price minus the down 
payment, instead of 53 percent of the selling price; 

° a correct procedure but work shown contains arithmetic errors; 

° the correct sum of the residual value and the down payment — $11,411; and 

° the correct difference between the residual value and the down 
payment — $8,411. 
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The sample “partial” response is an example of the correct difference between the 
residual value and the down payment. 

Smmple 66 pariiui" 




An “incorrect” response met none of the preceding criteria. The “incorrect” response 
shown below contains evidence of some understanding of the process for solving the problem. 
That is, it is not clear how the student arrived at 980.5, but if that were the residual value, when 
added to the down payment of 1,500 and subtracted from the selling price of 18,700, the answer 
obtained would be the 16,219.5 shown. The response nevertheless is scored as “incorrect.” 

Sample 66 incorrect 99 respemse 
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Information on students’ performance on this question is presented in Table 3.30. 
Thirty-four percent of student responses were scored as “complete,” and 31 percent were 
scored as “partial.” 
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Complete 


Partial 


Incorrect 


Omit 












All Students 


34 


31 


24 


8 


Males 


33 


28 


26 


10 


Females 


36 


34 


22 


6 


White 


39 


30 


22 


6 


Black 


17 


30 


34 


16 


Hispanic 


27 


36 


26 


7 


Asian/Pacific Islander 


42 


30 


18 


7 


American Indian 


* ★ ★ 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 



i 

NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 5. Use formula to find total cost. The fifth question is prefaced with requirements 
for the students about showing their work and explaining their reasoning. In the question, 
students were asked to calculate the total cost of a car given the following information: a 
formula for calculating monthly car payments, the selling price of the car, the down payment, 
the amount to be financed, the annual percent rate of the amount to be financed, and the length 
of the loan. Students also were given an approximate value to use for one of the quantities used 

( /xq\ *t3 

1 + ^ . This question was classified as an Algebra and Functions question 
and was designed to assess Problem-Solving ability. 



This question requires you to show your work and explain your 
reasoning. You may use drawings, words, and numbers in your 
explanation. Your answer should be clear enough so that another person 
could read it and understand your thinking. It is important that you show 
all of your work. 



5. A formula used to calculate a monthly car payment is 




where MP is the monthly payment, 

A is the number of dollars to be financed, 
r is the annual percent rate (e.g., if the annual percent rate is 
7%, then r is 0.07), and 
n is the length of the loan in months. 

Use the formula shown above to help calculate the total cost of 
purchasing an $18,000 car if there is a down payment of $4,000 and 
the remaining $14,000 is to be financed at an annual percent rate of 
8 percent for 48 months. 



Show the work that led to your answer. 

Note: In your calculations, use 1.3757 as the approximate value of 




0.08) 
12 ) 



48 
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This problem was difficult because students needed to determine how to organize and 
use the information provided in order to solve the problem. In other words, students needed to 
know how to use a formula with variables to compute the monthly payments and how to 
correctly substitute the approximate value provided in the “Note” into the formula. They also 
had to recognize that the formula is for a monthly payment; because the question asks for the 
total cost of purchasing the car, the monthly payment needs to be multiplied by 48 months and 
added to the down payment in order to arrive at the total cost. 

The responses to this question were scored with a 5-point rubric: “extended,” 
“satisfactory,” “minimal,” “partial,” and “incorrect.” The scoring rubrics took into 
consideration that the answers students gave may have differed because of rounding, although 
students were told at the beginning of the block that they should not round in their calculations 
until the final answer. Therefore, the following answers, if accompanied by evidence of correct 
work, were all scored as “extended” — $20,404.48; $20,404.41; $20,404.35; or $20,405.48. 
The sample “extended” response has the correct answer, $20,404.48 and shows the 
mathematical calculations done to arrive at the correct answer. The student first calculated the 
monthly payment, using the approximate value, 1.3757, in the calculations; then calculated the 
total payments by multiplying the monthly payment by 48 months; and, finally, calculated the 
total cost by adding the total of the monthly payments and the down payment. 

i SmmjpU® 
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A “satisfactory” response was one that produced an acceptable monthly payment shown 
in dollars and cents or produced a total cost derived from rounded values (e.g., using 0.0092 
for 0.0091713 in the numerator of the monthly payment formula). The sample “satisfactory” 
response provides an acceptable answer for the monthly payment, but fails to show the total 
cost of purchasing the car. If the student had completed the calculations, a correct answer 
would have been obtained. 

S&mple **&atis§s&et<&ry" 
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A “partial” response is one in which any of the following are correctly calculated and 
indicated: the monthly payment using rounded values; the numerator in the brackets, 
0.0091713; and/or the amount to be financed. The following “partial” response shows a 
rounded monthly payment of $342. However, unlike the “satisfactory” response, this response 
would have led to an incorrect final answer if the student had completed the calculations, due 
to the intermediate rounding up to $342. 




A “minimal” response is one that shows a correct substitution of the given values into 
the formula. In the “minimal” response shown below, the student rounded the monthly payment 
down incorrectly, made an error in multiplying that monthly payment by 48 months, and finally, 
added the selling price of the car to the monthly payments rather than the down payment, to 
arrive at a total cost for the car that was nearly double the $18,000 selling price. 
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An “incorrect” response met none of the criteria mentioned above. In the sample 
“incorrect” response shown below, the student relisted the information provided but did not 
attempt any calculations with that information. 

SwdMjpU<E 66 nm(E(iDd o d o (B(Ed 99 d'cBsjpoDmscB 




The data on student performance on this question are presented in Table 3.31. 
Twenty-three percent of the responses were scored at least “satisfactory,” but 44 percent of the 
responses were scored as “incorrect.” 
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Extended 


Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


US 














All Students 


9 


14 


7 


12 


44 


9 


Males 


9 


12 


7 


10 


45 


12 


Female 


8 


16 


8 


14 


43 


7 


White 


10 


16 


8 


12 


40 


8 


Black 


2 


4 


4 


12 


54 


19 


Hispanic 


4 


1 1 


4 


13 


55 


8 


Asian/Pacific Islander 


19 


15 


8 


10 


38 


8 


American Indian 

i 


* * * 


* * * 


* * * 


* * * 


* * * 


* * * 



NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 
***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Question 6. Find amount saved if leased. As students encountered Question 6, they were 
provided with information that they were told would be relevant for Questions 6 and 7. This 
information included formulas for computing the cost of leasing or buying a particular car and 
for determining the amount of money for which the car could be resold after 4 years. 

In the sixth question, students were told that Mary bought a car and sold it after 
4 years. They were asked to determine the amount of money Mary could have saved if she had 
leased the car for 4 years instead of buying it. This question was designed to assess content 
from the Algebra and Functions strand and Procedural Knowledge. Students were instructed to 
show how they solved the problem. 



Questions 6-7 refer to the following information. 

Mary is interested in leasing or buying a particular car model. She has 
determined that the cost of leasing this car for 4 years is 0.78/? + 450, 
where p is the selling price. Mary has also determined that the cost of 
buying this car, including finance charges, is 1 .2 p + 80. At the end of 
4 years, when the car is paid off, it can be sold for 0.4/?. 

6. Mary decided to buy the car at a selling price of $20,000. She kept the 
car for 4 years and then sold the car to recover some of the costs. If 
Mary had decided to lease the same car for 4 years, what amount of 
money would she have saved? Show the work that led to your 
answer. 



To solve the problem, students needed to understand how to use the formulas provided, 
correctly substituting in the selling price of $20,000. Computing the cost of leasing is relatively 
straightforward, 0.78(20,000) + 450. However, computing the cost of buying involved adding 
the finance charges, 1.2(20,000) + 80, and also subtracting out the amount for which the car can 
be sold at the end of 4 years, 0.4(20,000). Finally, students had to subtract the cost of leasing 
from the cost of buying and reselling to get the answer of $30, the amount saved by leasing. 
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The responses to this question were scored on a 4-point rubric: “satisfactory,” “partial,” 
“minimal,” and “incorrect.” Responses scored as “satisfactory” had the correct number of 
dollars saved, $30, and showed an acceptable procedure for getting to that answer. The sample 
“satisfactory” response shows the correct procedure for calculating the cost of leasing, then the 
cost of buying and reselling, and, finally, the correct savings of $30. 
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“Partial” responses satisfied one of the following criteria: 



O both costs were correctly calculated, but the amount saved with leasing was 
not specified; 

O the cost to buy the car was calculated without subtracting out the amount gained from 
reselling; or 

O the correct amount saved, $30, was indicated, but the work involved in calculating 
that number was not shown. 

The sample “partial” response shown below includes the correctly calculated cost of 
leasing and buying the car but fails to take into account the amount gained from reselling the 
car; it therefore gives $8,030 as the amount saved by leasing. 

SmmjpUcB 66 p(BQ°ttfmU 99 v°<b§ip(idm§(B 
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A “minimal” response is one in which the cost of either leasing or buying is correctly 
calculated, or the cost to buy is correctly calculated but the gain from reselling is not taken into 
consideration. The sample “minimal” response correctly calculates the cost of buying, and 
although it has the correct formula for leasing, incorrectly calculates the cost of leasing. 
Furthermore, in determining the dollars saved by leasing, the student failed to take into 
account the amount gained from reselling. 

SaadmjpUxB 66 vommnm(mU 99 o°®s]p®oos© 




An “incorrect” response satisfied none of the criteria mentioned above. The following 
“incorrect” response correctly calculates a part of the formula for the cost of leasing, but goes 
no further. 
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The information presented in Table 3.32 shows that the students did relatively well on 
this question. Twenty-seven percent of the responses were scored as “satisfactory,” and 
23 percent were scored as “partial.” This means that students at least were able to calculate the 
cost of leasing and buying; in some cases the failure to include the gain from the resale was 
probably an oversight rather than inability to understand how to solve the problem. 
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All Students 


27 


23 


20 


21 


5 


Males 


27 


19 


18 


24 


7 . 


Females 


27 


27 


21 


18 


3 


White 


33 


24 


20 


16 


4 


Black 


6 


24 


19 


36 


1 1 


Hispanic 


12 


23 


22 


30 


7 


Asian/Pacific Islander 


29 


16 


20 


32 


2 


American Indian 


★ * * 


★ ★ ★ 


■k -k -k 


★ ★ ★ 


* * * 


i 



NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 

***Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 

Question 7. Price lease vs. buy. In the seventh question, students were asked to find the 
selling price of a car for which the leasing cost for 4 years and the buying cost with a resale 
after 4 years are the same. As in the other questions in the question block, students also were 
asked to show how they got their answer. This question was classified as an Algebra and 
Functions question designed to assess Procedural Knowledge. 



7. For the particular car model that Mary is interested in, determine the 
selling price for which the car costs the same to buy and resell at the 
end of 4 years as it costs to lease the car for 4 years. Show the work 
that led to your answer. 
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To answer this question most efficiently, students had to know how to set up an equation 
that equated the cost of leasing with the cost of buying and reselling, and to solve for the selling 
price of the car that would satisfy that equation. The responses to this question were scored 
using a 4-point rubric: “satisfactory,” “partial,” “minimal,” and “incorrect.” A “satisfactory” 
answer is one in which the students find the correct selling price of $18,500 either through 
setting up and solving the correct equation or through trial and error. In the sample 
“satisfactory” response shown below, the student wrote out the correct equation and calculated 
the selling price correctly. The student also included computations to show that, with the 
selling price of $18,500, both buying and leasing the car would cost $14,880. 

Smmple 6 S sm iisimc iordy" response 
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Responses scored as “partial” were those in which the correct equation was written out, 
but not solved, or the correct selling price was obtained, but the work contained a 
computational error. In the “partial” response shown, the correct answer is obtained, but there 
appears to be some carelessness in writing out the computations. For example, in the original 
equation, the student shows the resale price as being added to the selling price rather than 
subtracted from it. However, in the next line of the solution, the student has correctly 
subtracted the resale price after all. 

Sample 




“Minimal” responses included an equation without the resale price and therefore 
arrival at a wrong answer; or use of the resale price, 0.4p, incorrectly in the algebraic 
relationship. In the following “minimal” response, the student confused the equations for 
leasing and buying the car, although the resale proceeds were correctly subtracted out from 
what the student labeled as the cost of buying the car. The calculations were carried out 
correctly but, of course, the wrong answer was obtained. 

Sample ** minimal * 5 response 
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An “incorrect” response is one in which none of the above criteria were satisfied. The 
sample “incorrect” response used the selling price of the car mentioned in Question 6 and 
therefore appeared to show that the student did not understand how to approach solving 
this problem. 
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Information on student performance on this item is presented in Table 3.33. 

Sixteen percent of the responses were scored as “satisfactory,” while 57 percent were scored 
“incorrect.” Student responses were more likely to be scored “satisfactory” than “partial” or 
“minimal,” suggesting that, if they understood the question at all, they were likely to cany 
through to the final solution correctly. 
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All Students 


16 
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57 
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Males 


17 
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52 


1 1 


Females 


16 


1 


7 


62 


6 


White 


19 
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8 


55 
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Black 


4 


1! 


4 


59 


19 


Hispanic 


7 


0! 


3 


66 


9 


Asian/Pacific Islander 


18 


3 


14 


57 


2 


American Indian 


* * * 


* ★ ★ 


★ ★ ★ 


★ ★ ★ 


* * * 
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NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

***Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The fourth-grade released Theme block was about planning a Butterfly Booth for the school’s 
science fair. Of the six questions posed, one was classified as measuring Procedural 
Knowledge, the remaining five were classified as measuring Problem-Solving abilities. 
Questions were designed to assess content in four content strands: Number Sense, Properties, 
and Operations; Measurement; Geometry and Spatial Sense; and Algebra and Functions. 

In addition to solving the problems, students were generally asked to provide explanations of 
their answers, using mathematical computations, drawings, or words. 

The following are findings of what students appear to know and are able to do based on 
their performance on the questions in the Butterfly Booth block: 

O A majority of fourth-grade students appeared to be familiar with centimeter 
measurements and were able to measure using a ruler marked off in centimeters. 

A substantial proportion of students, however, appeared not to understand what it 
means to measure to the nearest centimeter. 

O In most of the questions, students were required to solve a multistep problem. 
Students appeared to have difficulty with multistep problems, even those that 
required straightforward calculations at each step of the problem. 

O Students appeared either to not understand what is meant by the word “symmetrical” 
or not understand how to use grid paper to show symmetry in a drawing. 

O Students appeared to be familiar with using proportions in reasoning but, in 
many cases, they had difficulty carrying out proportional operations with 
acceptable precision. 

O With the exception of the first question, most students attempted to answer the 

questions posed, even though large percentages produced responses that were scored 
as “incorrect.” 

O Although students’ explanations were often incomplete, most students attempted to 
provide explanations for their answers. 

O In many cases, students seemed to lack the mathematical knowledge needed to solve 
the problems. In other cases, students appeared to understand the underlying 
mathematics but provided incorrect or incomplete responses as a result of 
carelessness, inexperience in writing out solutions to problems, or confusion over the 
wording of the question. 

The eighth-grade released Theme block was about building a doghouse. Of the 
10 questions that students encountered in this block, 4 were designed to assess Conceptual 
Understanding, 1 to assess Procedural Knowledge, and 5 to assess Problem-Solving ability. The 
questions were classified in the content strands of Number Sense, Properties, and Operations; 
Measurement; Geometry and Spatial Sense; and Algebra and Functions. In three of the 
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questions, students were asked to write an explanation of how they solved a problem or to 
describe a mathematical operation. The following are findings of what students appear to know 
and are able to do based on their performance on the questions in the Doghouse block: 

° Many eighth-grade students appeared to understand how to inteipret and use 
linear dimensions. 

° Some students showed understanding of the use of scales and ratios; however, for 
many students this understanding was not fully developed. 

° Some students appeared to understand the relationship between perimeters and areas 
but had difficulty applying these concepts in a relatively complex problem. 

° Many students appeared to lack the depth of mathematical understanding required to 
plan and execute solutions to the problems. Others appeared to have the requisite 
conceptual and procedural knowledge, but were not careful in reading the problems 
or precise in carrying out the operations necessary for solving them. 

0 Omit rates for the constructed-response questions were much higher than those 
observed at grade 4, particularly for the two questions at the end of the block, which 
also happened to be among the most difficult. 

In the twelfth-grade released block, students were told that a family was considering 
whether to buy or lease a car. Most of the questions involved evaluating the costs of one of these 
options in relation to the other. Of the seven questions included in the block, six were designed 
to assess Procedural Knowledge and one to assess Problem-Solving ability. Four of the 
questions were classified as Number Sense, Properties, and Operations, and the other three 
were classified as Algebra and Functions. The following are findings based on student 
performance on the Buying-a-Car Theme block questions: 

0 Most students were able to solve problems involving simple computational skills. 
However, some of these students seemed to have difficulty with more complicated 
multistep problems, even those that required only simple computational skills at 
each step of the problem. 

° Many, but not nearly the majority, of twelfth-grade students appeared able to solve an 
algebraic problem if they were given the equation and values for variables in the 
equation. However, many students appeared to have trouble reading through a 
relatively complicated word problem in order to isolate the information needed to 
solve the problem completely. 

At all grade levels, the frequency with which students engaged in writing a few 
sentences about how to solve a mathematics problem, or engaged in writing reports or doing 
mathematics projects, was examined in relationship to student performance on the Theme 
blocks. Given the many recommendations to increase attention to extended-response problems 
in the classroom, it is perhaps significant that no positive relationship was seen between these 
two instructional practice variables and performance on the Theme blocks. 
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The NAEP 1996 assessment of mathematics included a special study to examine the 
performance of students at the eighth- and twelfth-grade levels who were taking or had taken 
advanced courses in mathematics (hereafter referred to as the Advanced Study). The motivation 
for the Advanced Study was an observation that the main NAEP assessment did not include 
enough advanced mathematics questions to allow students with appropriate preparation to 
demonstrate the full extent of their proficiency. 

For example, the main eighth-grade NAEP assessment covers the standard content of 
the basic eighth-grade curriculum. This means that most topics in the Algebra and Functions 
content strand of the NAEP framework currently are assessed at the pre-algebra level — that is, 
with limited use of literal terms to represent variables and limitations on the types of 
applications and formulas on which students can be assessed. However, a substantial percentage 
of eighth-grade students are currently enrolled in first-year algebra. The main NAEP assessment 
does not provide these students with sufficient opportunity to display what they know and are 
able to do in algebraic situations. A similar situation exists at the twelfth-grade level, where 
many students have taken pre-calculus or calculus, but are limited in what mathematical 
proficiency they can display on the main NAEP assessment. 

As a result, the National Assessment Governing Board (NAGB) and the National 
Center for Educational Statistics (NCES) decided to undertake a special study to assess what 
these “advanced” students could do as part of the NAEP 1996 mathematics assessment. 

At the eighth-grade level, the questions in the Advanced Study focused on algebra, with a 
special emphasis on examining what students know about various representations — graphical, 
numerical, symbolic, and written — for algebraic concepts and relationships, and the 
transformations among the various representations. At grade 12, the Advanced Study included 
a more even distribution of questions from all of the content strands of the NAEP 
mathematics framework. 
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To qualify for the Advanced Study, eighth-grade students had to be currently enrolled in or 
already have taken first-year algebra or a more advanced course such as geometry. Twelfth-grade 
students had to be enrolled in or have already taken a pre-calculus course, a course equivalent 
to pre-calculus, or a more advanced course such as calculus. These qualifying students 
represented 21 percent of the student population at grade 8 and 24 percent of the student 
population at grade 12. Students at participating schools who qualified but were not selected for 
the Advanced Study were eligible for the main NAEP assessment, where their performance 
could be directly compared to that of students with less rigorous preparation. (Appendix A 
provides more detail on the sampling for the Advanced Study.) 

At both grade levels, each student in the Advanced Study completed a special NAEP 
assessment booklet that consisted of three blocks of mathematics questions: two Advanced 
blocks and a block comprising questions selected from the main NAEP assessment. At the 
eighth-grade level, the questions from the main NAEP were mostly from the Algebra and 
Functions content strand, along with some questions from the Data Analysis, Statistics, and 
Probability strand. At the twelfth-grade level, the questions from the main assessment covered 
the more advanced content from the content strands of Algebra and Functions; Data Analysis, 
Statistics, and Probability; and Geometry and Spatial Sense. The block of questions from the 
main NAEP assessment was 15 minutes in length for both grade levels and was the first 
cognitive block in the Advanced Study assessment booklet. At the eighth-grade level, the two 
special blocks containing Advanced Study questions were each 20 minutes in length, whereas 
at the twelfth-grade level, the Advanced blocks were each 30 minutes long. All students 
participating in the Advanced Study were told they could bring their own calculators; students 
who did not bring a calculator were provided with a scientific calculator. 

[Enaplfott Mcttwaamascsdl Stlmdlnu 

§tw<M<Bm& IbmasIkggiPQDwmdil 

Table 4.1 compares the background characteristics of the grade 8 students in the Advanced 
Study with the characteristics of the students who took the national NAEP assessment but were 
not eligible for the Advanced Study. Table 4.2 includes additional data about students’ school 
types and their eligibility for income-related, school-based intervention programs. 

The data in Table 4.1 reflect some differences between students who qualified for the 
Advanced Study and those who did not. The percentage of female students was higher in the 
Advanced Study than among the students who were not eligible for the Advanced Study. 

In terms of racial/ethnic origins, greater percentages of White students and smaller percentages 
of Hispanic students were in the Advanced Study. In addition, Advanced Study students were 
more likely to have parents who had graduated from college. 



148 



R4dD 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 




THE NATION’S 



REPORT 



StoeJeffDi? Domogwiphk D’sslAvilmm, ©mde 8, U CAR0 



m 





©OcodlaDaDs 




Advanced Study 


Not Eligible for 
Advanced Study 


Gender 






Males 


48 


53 


Females 


52 


47 


Students who Indicated Their 
Race/Ethnicity as... 




62 


White 


71 


Black 


14 


18 


Hispanic 


6 


13 


Asian/Pacific Islander 


6 


6 


American Indian 


2 


2 


Students who Reported Their 
Parents' Highest Level of 
Education as... 






Did Not Finish High School 


3 


8 


Graduated From High School 


14 


23 


Some Education After High School 


20 


19 


Graduated From College 


58 


38 


1 Don't Know 


5 


12 


Home Environment Contains... 






0-2 Types of Educational Materials 


12 


23 


3 Types of Educational Materials 


26 


32 


4 Types of Educational Materials 


62 


45 


Students From... 






Northeast 


27 


18 


Southeast 


17 


22 


Central 


30 


28 


West 


26 


31 


Students Live in... 






Center City 


36 


34 


Urban Fringe/Small City 


39 


35 


Rural 

1 


25 


31 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Paralleling the findings for parental education, the data on types of educational 
materials in the home (newspapers, books, encyclopedias, magazines, etc.) show that students 
included in the Advanced Study were more likely to have 4 such materials in their homes, 
while students not eligible for the Advanced Study were more likely to have 0-2 or 3 of these 
materials in their homes. 

There were no differences in the proportions of Advanced Study or not eligible students 
based on region or type of community. 

<^(Bm<n>gjv°(ia]pBan<£s 

The data in Table 4.2 describe the types of schools students attended and present categorical 
information about special programs within schools for which the students may have been 
eligible. The data for type of school do not show any differences in the proportions of students 
participating in the Advanced Study relative to those eighth-grade students who were not 
eligible. However, the data for special programs do show differences; specifically, fewer 
students in the Advanced Study participated in Title I programs or were eligible for the federal 
Free/Reduced-Price Lunch program. 
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Advanced Study 
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Public Schools 


84 


90 


Nonpublic Schools 


16 


10 


Title 1 Participation... 






Participated 


3 


14 


Did Not Participate 


97 


86 



Free/Reduced-Price Lunch 
Program Eligibility... 

Eligible 
Not Eligible 
Information Not Available 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 





0 

ERIC 



R42 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



CEwsst*©©®® emphases 

NAEP also collected information from students’ teachers concerning the instructional emphases 
they placed on content areas in the curriculum and on developing students’ mathematical 
process abilities. Questionnaires were used to collect this information* as well as information on 
the use of calculators during mathematics instruction. Data from the teachers’ responses are 
shown in Tables 4.3 through 4.5. The percentages in the tables reflect the percentages of 
students whose teachers responded as shown. 

The data in Table 4.3 indicate that eighth-grade students in the Advanced Study were 
more likely to receive “some” emphasis on the area of Numbers and Operations and less likely 
to receive “a lot” of emphasis on this area than students who were not eligible for the Advanced 
Study. In addition, the Advanced Study students were more likely to receive “a little” emphasis 
on Measurement in their programs and less likely to receive “a lot” of emphasis on Geometry 
and Spatial Sense. There were no differences in student exposure to the area of Data Analysis, 
Statistics, and Probability. 

Grade 8 students in the Advanced Study also were reported as receiving more emphasis 
in the area of Algebra and Functions than students not qualifying for the Advanced Study. This 
is consistent with the curricular prerequisites on which study selection was based. 




151 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in : Mathematics 



143 




Emph(o}§®§ m €B@§§<&§, 



THE MAHON’S 
REPORT 



CARD 



i & r IWWIIMMiy. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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In This Mathematics Class 
How Often Do 
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Numbers and Operations 










Advanced Study 


0! 


6 


21 


73 


Non-Eligibles 


0! 


1 


9 


90 


Measurement 










Advanced Study 


4 


39 


45 


13 


Non-Eligibles 


1 


19 


59 


21 


Geometry 










Advanced Study 


6 


26 


54 


14 


Non-Eligibles 
Data Analysis, 


2 


20 


52 


26 


Statistics, and Probability 










Advanced Study 


9 


33 


46 


12 


Non-Eligibles 


6 


30 


48 


16 


Algebra and Functions 










Advanced Study 


0! 


1 


7 


92 


Non-Eligibles 

I • 


2 


9 


40 


49 
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The data in Table 4.4 relate to four general mathematical processes: problem solving, 
reasoning, connections, and communication. The data indicate that the students in the 
Advanced Study received much the same instructional emphases as other grade 8 students on 
learning mathematical facts and solving routine problems. However, the students qualifying for 
the Advanced Study were given more emphasis on solving unique problems and communicating 
ideas in mathematics. This finding substantiates the opportunity-to-learn differences that arise 
from the curricular tracking of students in the upper middle grades/junior high schools. 1 



Kifer, E. (1993). Opportunities, talents, & participation. In L. Burstein (Ed.), The IEA Study of Mathematics III : 
Student growth and classroom processes (pp. 279-308). New York: Pergamon Press; Schmidt, W. H., et al. (1998). 
Facing the consequences: Using T1MSS for a closer look at United 'SttBesO/iathematics and science education. Dordrecht, 
Netherlands: Kluwer Academic Publishers. dL tj 
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In This Mathematics Class 
How Often Do 
You Address... 










Learning Mathematical 










Facts and Concepts 










Advanced Study 


2 


7 


13 


78 


Non-Eligibles 


0 


5 


16 


79 


Learning Skills and 










Procedures Needed to 
Solve Routine Problems 










Advanced Study 


0! 


4 


13 


82 


Non-Eligibles 


1! 


1 


18 


80 


Developing Reasoning and 
Analytical Ability to Solve 










Unique Problems 
Advanced Study 


0! 


5 


28 


68 


Non-Eligibles 


1 


8 


44 


46 


Learning How to 










Communicate Ideas in 
Mathematics Effectively 










Advanced Study 


0! 


10 


34 


55 


Non-Eligibles 


1 


17 


43 


39 


1 



NOTE: Row percentages may not total 100 due to rounding. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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NAEP also collected information on the extent to which students use calculators as part of 
their normal mathematics experience in schools. Table 4.5 contains the data reported by 
grade 8 teachers relative to student access to and use of calculators as part of their regular 
instructional programs. The data reflect that students in the Advanced Study had fewer 
restrictions on the use of calculators in their regular school mathematics programs than 
students not eligible for the Advanced Study. Similar results were found regarding the use of 
calculators on classroom tests; more than 80 percent of the Advanced Study students, compared 
to about two-thirds of the non-eligible students, had teachers who allowed calculators to be 
used in these circumstances. Responses to questions dealing with access to school-owned 
calculators and with instruction on the use of calculators, however, showed no differences 
between eighth-grade students in the Advanced Study and those who were not eligible for 
the study. 
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Advanced Study 
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Non-Eligibles 


42 
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Do You Permit Students in This Class 






to Use Calculators for Tests... 






Advanced Study 
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18 
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36 


Do the Students in This Class 






Have Access to Calculators 






Owned by the School... 






Advanced Study 
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Non-Eligibles 
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Do You Provide Instruction to 
Students in This Class in the 






Use of Calculators... 






Advanced Study 


80 


20 


Non-Eligibles 


84 


16 
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SOURCE: Notional Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The results from the Advanced Study did not lend themselves to either the development of a 
separate proficiency scale or equating to the main NAEP mathematics scale (see Appendix A). 
However, as noted above, some of the students who were eligible for the Advanced Study 
actually participated in the main NAEP assessment. This affords the opportunity to compare 
directly the performance of students who were eligible for the Advanced Study and those who 
were not eligible, using the composite NAEP scale. The data in Table 4.6 show that students in 
the eighth grade who were eligible for the Advanced Study had an average NAEP score of 
300, while non-eligible students had an average score of 264. 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The two blocks of Advanced Study questions at the eighth-grade level consisted of 10 and 
12 questions, respectively. The questions were classified primarily in the Algebra and 
Functions content strand of the NAEP mathematics framework, although a few questions 
designed to assess concepts and procedures from the Measurement and Data Analysis, 
Statistics, and Probability strands also were included among the 22 questions. Furthermore, 
some of the questions classified as Algebra and Functions were open to solutions via Number 
Sense, Properties, and Operations approaches. 

Six of the algebra questions required students to make an interpretation or complete an 
operation based on understanding symbolic representations of algebraic relationships. Nine of 
the questions required students to begin by considering written descriptions of algebraic 
relationships, possibly translate these relationships into numerical or algebraic representations, 
and then find solutions for the situations described. Another question was designed to measure 
students’ understanding of and ability to use the Pythagorean theorem in an applied setting. 
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Table 4.7 displays the breakdown of questions in the Grade 8 Advanced Study by 
content strand and response structure. As noted previously, the Advanced Study at grade 8 
differed from the main NAEP assessment in the distribution across content strands. 2 
In addition, the Advanced Study contained a far greater percentage of questions that required 
students to show or explain their work than were found in the main NAEP. 
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Properties, and Operations 
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Measurement 
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Geometry and Spatial Sense 


1 
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Algebra and Functions 
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Multiple-Choice 


9 


Short Constructed-Response 


1 1 


Extended Constructed-Response 


2 
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SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



2 Reese. C. M., Miller, K. E., Mazzeo, J., & Dossey, J. A. (1997). NAEP 1996 mathematics report card for the nation and 
the states . Washington, DC: National Center for Education Statistics. 
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Students’ overall performance on the Advanced Study, measured by average percentage correct 
scores, is presented in Table 4.8. Students’ average percentage correct score was 36 percent, 
and male students outperformed females. White, Hispanic, and Asian/Pacific Islander students 
outperformed Black students, and White and Asian/Pacific Islander students also outperformed 
Hispanic students. The sample of American Indian students was too small to permit reliable 
estimates of the performance on the Advanced Study blocks or on individual questions. 
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All Students 
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Males 


38 


Females 


34 


White 


40 


Black 


19 


Hispanic 


28 


Asian/Pacific Islander 


50 


American Indian 


★ ★ ★ 
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*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



Examples of Advanced Study questions and 
student performance 

Some sample questions from the Grade 8 Advanced Study are presented below to show how 
well the Advanced Study students could perform on specific tasks and to explicate the types of 
knowledge and skills students at the eighth-grade level needed to have to solve these problems. 
Information on how students performed on these questions is presented in terms of percentages 
correct on the individual questions. Performance information is provided for all students and by 
gender and race/ethnicity subgroups. 
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The first three sample questions constitute a family of questions based on a car wash 
being conducted by an eighth-grade class. The problem stimulus provided students with 
important data in various forms: written, graphical, and symbolic. Based on this information, 
students were asked to complete three different short constructed-response questions. 
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The first question asked students to interpret the information in the graph to find the 
fixed costs, or amount spent on supplies, before the first car was washed. In order to answer the 
question correctly, students needed to consider the y-intercept on the graph (i.e., the profit 
value associated with the situation before the first car was washed). They could do this either 
by inspecting the graph and finding the y-intercept at y = - 8, or by substituting x = 0 into the 
equation shown in the stimulus and solving to find y = - 8. 



4. According to the graph, how much money did the class spend on car 
wash supplies? 



Explain how you found your answer. 



Student responses were rated “correct,” “partial,” or “incorrect.” The scoring rubric for 
the question required that students’ responses indicate an understanding that initially the 
number of cars washed was zero (i.e., that the y-intercept must be used). That is, to receiver 
rating of “correct,” students were required to give both the correct response of 8 and an 
explanation that showed a correct understanding of the y-intercept in the graph or in the 
equation. Two sample “correct” responses are shown below. These responses clearly indicate 
both a correct answer and a rationale for why that answer is correct. The first response 
indicates recognition of the values associated with zero cars washed. The second response 
contains evidence of understanding the meaning of the y-intercept in the symbolic equation 
presented in the stimulus of the problem. 

&umpU@ ^^orrect" resp®m§® U 



4, According to the graph, how much money did the class spend on car 
wash supplies? 



^ D Q-U-fegS 



Explain how you found your answer. 
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4. According to the graph, how much money did the class spend on car 
wash supplies? 

^.00 




Explain how you found your answer. 
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Students responding with the value 8 and no explanation or an incorrect explanation or 
students responding without a value or with an incorrect value but with an explanation that 
indicated that they knew that the value of the y-intercept answered the question received 
“partial” ratings. The sample “partial” response shown below indicates a correct value, but 
lacks an acceptable rationale for the answer. 
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4. According to the graph, how much money did the class spend on car 
wash supplies? 



Explain how you found your answer. 
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Students simply responding - 8 for the amount spent on car wash supplies received a 
rating of “incorrect.” Student responses that contained an incorrect value and an explanation 
that indicated a lack of understanding that the answer should be found in the value of the 
y-intercept also received an “incorrect” rating. The “incorrect” response shown demonstrates 
such a lack of understanding. 



4. According to the graph, how much money did the class spend on car 
wash supplies? 

$ 9.35 



Explain how you found your answer. 
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Table 4.9 presents the breakdown of student performance according to gender and 
race/ethnicity. Overall, 13 percent of the student responses were rated “correct,” 26 percent 
were rated “partial,” and 36 percent were rated “incorrect.” Twenty-one percent of the students 
did not respond to the question, and an additional four percent, not shown in the table, only 
wrote off-task remarks on their papers. 3 



3 Student responses for this and all other constructed-response questions were scored as “off task” if the student 
provided a response that was deemed not related in content to the question asked. There are many examples of these 



types of responses, but a simple one would be “I don’t like this test.” 
valid attempts to answer the question that were simply wrong. 



In contrast, responses scored as “incorrect” were 
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All Students 


13 


26 


36 
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20 


Females 


10 


23 


41 


21 


White 


15 


30 


34 


19 


Black 


3! 


14! 


46! 
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Hispanic 


10 


23 


40 


23 


Asian/Pacific Islander 


14 


33 


35 


17 


American Indian 
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* * * 


★ ★ ★ 


★ ★ ★ 



NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



The second question in this three-question family asked students to find the number of 
cars that must be washed before the class would start to earn a profit. In order to answer the 
question correctly, students needed to find the value of x for which y = 0. This value could be 
found by locating the ^-intercept of the graph (i.e., x = 4), or by solving y = 2x - 8 for y - 0. 
Some students also responded with the value of 5 cars, as this is the first point at which there is 
a positive profit. Values of either 4 or 5 were accepted as correct numerical answers for the 
value portion of the question, and correct corresponding explanations were accepted for the 
explanation portion of the question. 



5, How many cars do the students have to wash before the class would 
start to earn a profit? 



Explain your answer. 
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Student responses were rated as “correct,” “partial,” or “incorrect” according to the 
following rubric, or rating scale. To receive a rating of “correct” a response had to contain both 
the correct answer and an explanation of why it was the right value. Responses received a rating 
of “partial” when they contained a numerical answer of 4 or 5 without a supporting explanation, 
or when they contained a number other than 4 or 5 but demonstrated an understanding that the 
%-intercept must be used. Responses were rated “incorrect” when students failed to give either 
the correct answer or an explanation of how to get a correct answer. 

The “correct” response shown below reflects the work of a student who recognized that 
washing four cars would pay for the supplies and that washing a fifth car would be needed to 
achieve a positive profit. 
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§. How many cars do the students have to wash before the class would 
start to earn a profit? 
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The “partial” response that follows shows a correct numerical answer but has an 
explanation that contains errors. 
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5 . How many cars do the students have to wash before the class would 
start to earn a profit? 




Explain your answer. 
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Information on student performance is presented in Table 4.10. Overall, 21 percent of 
students submitted responses rated “correct,” 19 percent submitted responses rated “partial,” 
and 39 percent turned in responses rated “incorrect.” Another 18 percent of the students 
omitted the question. Performance on this question was somewhat better than performance on 
the previous problem. In this case, compared to the preceding problem, students had to make 
less of a conceptual leap to interpret what the problem was asking them to do. The problem also 
was more open to being solved through numerical reasoning, 2 X [ ] = 8, and therefore, was 
more straightforward than the fixed-costs problem. There was some ambiguity in the phrasing 
of the question, however, which required acceptance of either 4 or 5 as the correct answer. 
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NOTE: Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate, 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment, 



The third question in the family, also rated on a three-point scale, asked students to 
find the greatest profit (in dollars) that the class could expect to make from the car wash. To 
solve this problem, students needed to find the profit associated with x = 50, the maximum 
number of cars that could be washed. By substituting into the equation y - 2x - 8, they could 
find that the maximum profit that could be expected was $92. 



6. What is the greatest profit (in dollars) that the class can expect to 
earn? 



Show your work. 
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To receive a rating of “correct,” a student only had to give the correct answer of $92. A 
supporting rationale was not required for the response to be rated “correct” even though the 
question asked students to show their work. The scoring rubric allowed raters to give partial 
credit for an incorrect value if the correct process was shown. While many of the students who 
received full credit simply wrote in $92 or showed something like 2(50) - 8 = 92, the “correct” 
response shown below is from a student who clearly communicated his or her approach as well 
as giving the correct solution. 
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Students who used the equation and made a correct substitution, but made an error in 
calculation, were awarded a rating of “partial.” For example, some divided the 92 by 2 or 
incorrectly multiplied 2 X 50 to get 200 and then subtracted 8 to arrive at an answer of 192, as 
in the “partial” response shown. 
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6. What is the greatest profit (in dollars) that the class can expect to 
earn? 
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Show your work. 

y=^0O-1> 
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Student responses were rated “incorrect” if they neither found the value nor gave a 
method for finding that value. Some students receiving “incorrect” ratings omitted the start-up 
costs and simply multiplied 50 X 2 to arrive at an answer of $100; others attempted to estimate 
the answer, as shown in the following “incorrect” response. 
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Information on student performance is presented in Table 4.11. Overall, 27 percent of 
responses were rated “correct,” 4 percent were rated “partial,” and 40 percent were rated 
“incorrect.” Twenty-six percent of the students omitted the problem. On this question, a higher 
percentage of students submitted responses that were rated “correct,” but there was a slight 
decrease in overall performance (“correct” and “partial”) compared with the two previous 
questions. The increase in responses receiving a “correct” rating could be due to the fact that 
the student could read the problem and then simply substitute into the equation to find the 
answer. While the question is open to solution by numerical patterns or an extension of the 
graph, these are less likely approaches. It also is possible that the score distribution reflects 
the decision to give full credit for a correct numerical answer without requiring that work 
be shown. 
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NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 

The fourth sample question presented here for grade 8 is one that asked students to 
consider a situation in which one hot-air balloon is ascending and another balloon is 
descending. Students were given the rate of change in elevation and the initial starting point for 
each balloon and were asked to find the time required for the two balloons to reach the same 
altitude. The problem is most readily solved through the use of a single equation 3 1 = 1000 — 5 1 
or the system of equations: y = 3t and y = 1000 - 5 1. However, a student could solve the 
problem through a table of values and estimation or through experimentation. 



7. A hot-air balloon begins rising at the rate of 3 feet per second. At the 
same time, a second hot-air balloon that is 1 ,000 feet above the first 
balloon begins to descend at the rate of 5 feet per second. In how 
many seconds will the balloons reach the same altitude? 



Students’ solutions to this problem were rated on a three-point scale. “Correct” 
responses gave the correct numeric response of 125 seconds with or without an explicit 
rationale. Responses that reflected work that did more than restate the problem but failed to 
achieve a correct answer for the time in seconds were rated “partial.” The rating of “incorrect” 
was given to responses that contained neither a correct value nor a correct equation or 
formulation of the problem. 
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While many “correct” responses resulted from students simply solving 3 1 = 1000 - 5 1 , 
the sample “correct” response shows both the modeling that led to the development of the 
equation and the equation-solving approach employed. 
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7. A hot-air balloon begins rising at the rate of 3 feet per second. At the 
same time, a second hot-air balloon that is 1 ,000 feet above the first 
balloon begins to descend at the rate of 5 feet per second. In how 
many seconds will the balloons reach the same altitude? 
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The following “partial” response demonstrates some effort toward working out the 
problem but a failure to obtain the correct answer. 
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7. A hot-air balloon begins rising at the rate of 3 feet per second. At the 
same time, a second hot-air balloon that is 1,000 feet above the first 
balloon begins to descend at the rate of 5 feet per second. In how 
many seconds will the balloons reach the same altitude? 
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The results, presented in Table 4.12, show that 19 percent of the responses received a 
“correct” rating, 7 percent received a “partial” rating, and 58 percent received an “incorrect” 
rating. In addition, 13 percent of the students omitted the question. Students used a number 
of approaches in answering this question. While the use of a system of equations or a single 
equation that equated the two ways of representing height at the time desired, 3 1 = 1000 - 5 1, 
were most common, a number of numerical approaches were noted among the Advanced 
Study students. 
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NOTE: Raw percentages may nat total 100 due ta responses rated "Off Task" ar ta rounding, ar bath. 

*** Sample size is insufficient ta permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may nat be accurately 
determined and/arthe sampling distribution af the statistics does nat match statistical test assumptions (see Appendix A). 
SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The data in Table 4.13 describe the student background characteristics of the grade 12 
students in the Advanced Study compared to those of students in the main NAEP assessment 
who had not taken advanced courses and were therefore not eligible for the Advanced Study. 
Compared to the non-eligible group, higher percentages of Asian/Pacific Islander and White 
students and a lower percentage of Black students participated in the Advanced Study. In 
addition, students in the Advanced Study were more likely to have parents who had graduated 
from college. 

Similar to the findings at grade 8, the data on types of educational materials in the 
home (newspapers, books, encyclopedias, magazines, etc.) show that students included in the 
Advanced Study were more likely to have 4 such materials in their homes, while students not 
eligible for the Advanced Study were more likely to have 0—2 of these materials in their homes. 
There were no significant differences in Advanced Study participation based on region or type 



of community. 
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4 


7 


Graduated From High School 


13 


22 


Some Education After High School 


23 


26 


Graduated From College 


59 


, 41 


1 Don't Know 


2 


3 


Home Environment Contains... 






0-2 Types of Educational Materials 


13 


20 


3 Types of Educational Materials 


25 


27 


4 Types of Educational Materials 


62 


53 


Students From... 






Northeast 


25 


22 


Southeast 


21 


23 


Central 


28 


24 


West 


26 


30 


Students Live in... 






Center City 


32 


31 


Urban Fringe/Small City 


38 


38 


Rural 


29 


31 


[ 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



173 




Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



E ©5 



&&wdl<®Mit/®£lh(B)<mU QttcBmwggwmplhne® : 

Information on type of school and student participation in categorical programs is shown in 
Table 4.14. When these data are considered, more differences between the groups begin to 
emerge. A larger proportion of grade 12 students participating in the Advanced Study were 
from nonpublic schools than were students not eligible for the Advanced Study. The two groups 
did not differ in terms of participation in Title I programs, but fewer Advanced Study students 
were eligible for the federal Free/Reduced-Price Lunch program. 
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Advanced Study 


IhSot Eligible for 
Advanced Study 








Students who Attend... 






Public Schools 


82 


91 


Nonpublic Schools 


18 


9 


Title 1 Participation... 






Participated 


2 


2 


Did Not Participate 


98 


98 


Free/Reduced-Price Lunch 






Program Eligibility... 






Eligible 


7 


15 


Not Eligible 


56 


62 


Information Not Available 


37 


23 



[ 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Table 4.15 contains the data from teachers’ responses to questions requesting an indication of 
the amount of emphasis they placed on each of a number of subject matter content areas within 
the grade 12 curriculum. Note that the data shown here are only from the teachers who were 
teaching students in the Advanced Study; teachers of twelfth -grade students who took the main 
NAEP assessment were not surveyed. The data shown in Table 4.15 indicate that, in classes 
taken by Advanced Study participants, there was more “heavy emphasis” placed on Functions 
than on any other topic. Algebra and Trigonometry ranked next and did not differ much from 
each other in frequency of “heavy emphasis.” Geometry, at 23 percent, received “heavy 
emphasis” less often than Functions, Algebra, or Trigonometry, but more often than Statistics, 
Probability, or Discrete Mathematics. 
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Little 

Emphasis 


Moderate 

Emphasis 


Heavy 

Emphasis 


©uexate t)2 








How Much Emphasis on... 








Algebra 


4 


23 


73 


Geometry 


14 


63 


23 


Trigonometry 


5 


28 


67 


Functions 


2 


13 


85 


Statistics 


65 


25 


10 


Probability 


66 


25 


9 


Discrete Mathematics 


60 


34 


6 



SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



Classroom process emphases 

In Table 4.16, a second set of data from classroom teachers of Grade 12 Advanced Study 
students describes the emphases teachers reported placing on process-related activities. 
Specifically, teachers were questioned about four common mathematical processes: problem 
solving, reasoning, connections, and communication. Most students had teachers who reported 
that they placed “heavy emphasis” on the first three areas: facts and concepts, skills and 
procedures for solving routine problems, and reasoning and analysis for solving nonroutine 
problems. However, fewer students had teachers who reported a “heavy emphasis” on 
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preparing students to communicate the results of their mathematical endeavors. This latter 
finding is consistent with classroom practices data reported by twelfth-grade students who 
participated in the main NAEP assessment. 4 




m €Sm§es Taken by 
Grade 12, 1996 



THE NATION'S 
REPORT 
CARD 



m 



@0 XE&D D 2 

How Much Emphasis on... 

Learning Mathematical 
Facts and Concepts 

Learning Skills and Procedures 
Needed to Solve Routine Problems 

Developing Reasoning and 
Analytical Ability to Solve 
Unique Problems 

Learning How to Communicate 
Ideas in Mathematics Effectively 






Little 

Emphasis 


RHoderate 

Emphasis 


Heavy 

Emphasis 


2 


20 


78 


1 


22 


77 


2 


25 


73 


10 


47 


43 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



mma 3 msmagtB 

Data on calculator access and usage for students in the Advanced Study also were collected 
from their teachers. Table 4.17 presents findings about calculator access. The data reflect an 
even level of access to both scientific and graphing calculators in the classroom, but greater 
access to scientific calculators outside the classroom. From a different perspective, the data 
indicate that 80 percent of Advanced Study students were permitted unlimited access to 
calculators in their mathematics classes. However, the percentage of students having access to 
calculators during assessment sessions was somewhat lower, with teachers of 71 percent of the 
students reporting that their students always could use calculators on tests; the remainder of 
the teachers reported that their students were sometimes permitted to use calculators on tests. 



4 Reese, et al., (1997). op. cit. 
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Percentage o{ Stadenta 






Do You Permit Students in This Class 




Unrestricted Use of Calculators... 




Yes 


80 


No 


20 


Do Students in This Class 




Have Access to Scientific Calculators 




in Class... 




Yes 


83 


No 


17 


Do Students in This Class 




Have Access to Scientific Calculators 




Out of Class... 




Yes 


82 


No 


18 


Do Students in This Class 




Have Access to Graphing Calculators 




in Class... 




Yes 


84 


No 


16 


Do Students in This Class 




Have Access to Graphing Calculators 




Out of Class... 




Yes 


56 


No 


44 


Do You Permit Students in This Class 




to Use Calculators on Tests... 




Yes All 


71 


Yes Some 


28 


No 


0 


r ' 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Table 4.18 contains more detail about calculator instruction and graphing calculators. 

The responses indicate that more students received instruction in how to use graphing 
calculators than scientific calculators. The larger extent of training being devoted to graphing 
calculators may be due partially to their special uses in these classes and partially to their 
newness in grade 12 classrooms. 

Over 70 percent of the students in Advanced Study classrooms had full access to a 
graphing calculator for their study of mathematics, either from a complete set in the classroom 
or through a personally owned graphing calculator. Reportedly these calculators were primarily 
utilized for their unique capability — that is, graphing functions. 
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Percentage of Students 


®K3»)3 112 K 




In This Class Students are 




Provided Instruction in the Use of 




Scientific Calculators... 




Yes 


65 


No 


35 


In This Class Students are 




Provided Instruction in the Use of 




Graphing Calculators... 




Yes 


87 


No 


13 


Which of the Following Best Describes 
the Availability of Graphing Calculators 
in This Class... 




One 


1 


Less Than Six 


3 


Complete Set 


48 


Some Students Have One 


8 


Most Students Have One 


9 


All Students Have One 


24 


No Student Has One 


6 


If Graphing Calculators are Used 
in This Class, What is Their’ 
Primary Usage... 




Calculating 


7 


Graphing 


76 


Tables 


0 


Statistics 


1 


Symbolic Manipulation 


3 


Not Used 


13 


1 ■ ' - 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Table 4.19 compares performance on the composite NAEP mathematics scale for grade 
12 students in the main assessment who were, or were not, eligible for the Advanced Study. 

As can be seen, students who were eligible for the Advanced Study had an average scale score 
of 327, while non-eligible students had a lower average score of 297. 




Average Mathematics Sc®/® Scores by 
" '”7 ty for Advanced Study, Grade 12, 1996 



THE NATION’S 



REPORT 

CARO 



M 





Average Scale Score 






All Students 


304 


Eligible 


327 


Non-Eligible 


297 


i 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The two blocks of Advanced Study questions for grade 12 students each consisted of 
11 questions. Although the questions were advanced in nature, no questions required calculus. 
Most of the questions could be solved by several methods. 

The 22 questions were balanced among situations that were presented in symbolic, 
graphical, written, and visual formats. Many of the questions required students to make 
connections between two or more forms of representation in order to complete the requirements 
set by the question. 
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Table 4.20 displays the breakdown of questions by content strand and response 
structure. All five NAEP mathematics content strands were represented in the Advanced Study. 
Compared to the main NAEP assessment, however, the Advanced Study included considerably 
higher percentages of questions classified in the Algebra and Functions; Data Analysis, 
Statistics, and Probability; and Geometry and Spatial Sense strands; there also was a far greater 
percentage of questions that required students to show or explain their work. 5 




Distribution of Questions by Content Strand 
Response Format, Grade 1 2, 1996 



= THE NA TION'S 

REPORT 
CARD 





Humber of Items 


(§®oout3oaG > gGwsnxs) (fewecKagjs 




Number Sense, 
Properties, and Operations 


1 


Measurement 


2 


Geometry and Spatial Sense 


7 


Data Analysis, 
Statistics, and Probability 


5 


Algebra and Functions 


7 


Bseuxsaoes tfexMxsGs fee? Ousouoe 




Multiple-Choice 


7 


Short Constructed-Response 


10 


Extended Constructed-Response 

1 


5 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



IP<BfffI(iDV o m(&m(E(B dDm dikes MtMwc&mescBdl &dwdl / Q[i 

Students’ overall performance on the Grade 12 Advanced Study, measured by average 
percentage correct scores, is presented in Table 4.21. For the group as a whole, the average 
percentage correct score was 30 percent. Male students outperformed female students, and 
White and Asian/Pacific Islander students outperformed Black and Hispanic students. 

The sample of American Indian students was too small to permit reliable estimates of the 
performance on the Advanced Study blocks or on individual questions. Students who were 
currently taking mathematics or who were taking, or had taken, an AP course in mathematics 
outperformed students who were not currently taking a mathematics course or who had not 
taken an AP course in this subject area. 



5 Ibid. 
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Percentage Correct 


US 




All Students 


30 


Males 


32 


Females 


27 


White 


32 


Black 


14 


Hispanic 


19 


Asian/Pacific Islander 


32 


American Indian 


* * * 


Are Students Presently Enrolled 




in Mathematics... 




Yes 


30 


No 


18 


Are Students Presently Enrolled in or 




Have Previously Taken an 




Advanced Placement (AP) 




Mathematics Course... 




Yes 


37 


No 

I- 


26 

: iu 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 




182 



o 

ERIC 



nr 4 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



EssmmfpUcBS ©ff nmdInwndImmU qim<BSJtn<iDms mrndl stiw<M(Bmtt jpcBV°ff(iDV , mmm(E<B 
Four sample questions from the Grade 12 Advanced Study are presented below to show the 
types of knowledge and skills students at the twelfth-grade level needed to solve the Advanced 
Study problems. As before, student performance information is presented in terms of 
percentages correct on the individual questions. The subgroup comparisons for grade 12 
include gender, race/ethnicity, whether students are presently enrolled in mathematics, and 
whether students are enrolled in or have taken an AP mathematics course. 

The first sample question asked students to find a third value for a linear function, 

/, given two other values for the function. Students were asked to show their work. In order to 
solve the problem, students first had to realize that they were being asked to find the third point 
on a straight line, given two other points on the line. Students then could have proceeded to 
find the slope of the line connecting the two given points graphically or by using the equation 
for a straight line. The value of the slope, along with the coordinates of the given points, one of 
which is the y-intercept, allows for an algebraic construction of an equation for the line. Solving 
this equation for the value x = 3.8 results in the correct answer of 1.58 for/3.8). Other 
students might have approached the problem graphically and more geometrically and used 
proportions such as 0.35/1 = hi 3.8 to find h, which they then added to the value of the 
y-intercept to get the required value of 1.58. 



3. If /is a linear function such that /( 0) = 0.25 and/(l) - 0.6, what is the 
value of/(3.8)? Show your work. 



Student responses were rated “correct,” “partial,” or “incorrect.” Responses of 1.58 or 
1.6 with correct accompanying work were rated “correct.” If no work was shown, the answer 
had to be 1.58 in order to be considered “correct.” Responses were rated “partial” if they 
contained some form of the equation y = 0.35% + 0.25 or correctly found the values of 
J[ 3) = 1.3 and/4) = 1.65 or correctly found the slope, m = 0.35. All other responses were 
rated “incorrect.” 
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One example of a correct” student response is shown below. Here the student correctly 
calculated the slope and then used the slope value and one of the given ordered pairs to find an 
equation of the line. Finally, the value of /(3.8) was computed correctly. 

SaamjpU<s 66 (e<idv°v°(B(£11 99 




One response that received a “partial” rating follows. The student correctly calculated 
the slope as 7/20 but did not complete the problem. 
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If/is a linear function such that /( 0) = 0.25 and /(l) = 0.6, what is the 
value of /(3.8)? Show your work. 



& 



X - 
' h 

\ °Q 



2. 

2o 

1 



to 
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The data in Table 4.22 reflect student performance on the linear functions question. 
Twenty percent overall submitted a response rated “correct,” and another 11 percent received 
partial credit. This performance is somewhat disappointing, given that the question covers 
basic concepts of linear functions. Common errors included reversing values for the 
independent and dependent variables, perhaps indicating a fundamental misunderstanding of 
the functional notation, and reversing the definition of a slope. Some students incorrectly 
defined the slope as the change in x over the change in y. 
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Correct 


Partial 


Incorrect 


Omit 












All Students 


20 


1 1 


43 


23 


Males 


25 


10 


38 


23 


Females 


16 


1 1 


48 


23 


White 


23 


1 1 


42 


21 


Black 


4 


8 


44 


43 


Hispanic 


11! 


6! 


46! 


32! 


Asian/Pacific Islander 


22 


12 


42 


22 


American Indian 


* * * 


* * * 


* * * 


* * * 


Are Students Presently 
Enrolled in Mathematics... 










Yes 


21 


11 


43 


23 


No 


9! 


4! 


46! 


37! 


Are Students Presently 
Enrolled in or Have 
Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 










Yes 


31 


12 


38 


17 


No 


13 


10 


46 


27 


1 ! 











NOTE; Row percentages may not total 1 00 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The second sample question from the Grade 12 Advanced Study asked students to rank 
order the volumes of three square pyramids from smallest to largest and supply a rationale for 
their ranking. As data, students were given one of the measurements of altitude, slant height, or 
lateral edge for each pyramid and told that all pyramids resided on a square base with sides of 
10 units. Students also were given the general formula for the volume of a pyramid. 



This question requires you to show your work and explain your reasoning. 
You may use drawings, words, and numbers in your explanation. Your answer 
should be clear enough so that another person could read it and understand 
your thinking. It is important that you show all of your work. 




10. The figure above shows the altitude, lateral edge, and slant height of 
a square pyramid. Each of three square pyramids P, Q, and R (not 
shown) has a base with side 10. In each pyramid the lateral edges are 
of equal length. Pyramids P, Q, and R have the following 
characteristics. 

a. The altitude of pyramid P is 10. 

b. The lateral edge of pyramid Q is 10. 

c. The slant height of pyramid R is 10. 

List the pyramids by order of volume from smallest volume to largest 
volume. 

Support your conclusion with mathematical evidence. 

[The formula for the volume of a pyramid with base area B and 
height (altitude) h is V = j Bh .] 
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Since the area of the base was the same for all three pyramids, the solution to the 
problem was dependent on the height, or altitude (/i), of the pyramids. The height of pyramid P 
was given as 10 units. The height of pyramid R could be found by recognizing that the distance 
from the foot of the segment representing the slant height to the middle of the base was 5 units 
and then using the Pythagorean theorem to find the height of the pyramid, namely 5V3, or 
approximately 8.66 units. The height of pyramid Q also could be found by using the 
Pythagorean theorem. In this case, however, students had first to solve for either the slant 
height or the length of the segment running from the base of the lateral edge to the foot of the 
altitude. Once either of these dimensions had been determined, it was possible to solve for the 
altitude, or height. The correct solution was 5^2, or approximately 7.07 units. 

Since all three pyramids had the same base area, and their volumes could be found by 
multiplying the area of the bases by one-third of the heights, the volumes were ordered from 
smallest to largest in the same order as the heights were ordered from shortest to longest. That 
is, Vp < V R < V,,, since h Q <h R < h p . 

A few students noted that there is a quicker way to solve the problem without doing any 
calculations. Because the hypotenuse of a right triangle is the side of greatest length, it can be 
argued that the lateral edge is greater than the slant height, which in turn is greater than the 
altitude. Consequently, the pyramid with a lateral edge of 10 is “shorter” than the pyramid with 
a slant height of 10, which is “shorter” than the pyramid with an altitude of 10. Since “shorter” 
relates to the height of the pyramids, and since all of the pyramids are on the same base, the 
correct ordering follows. 

Responses were rated “extended,” “satisfactory,” “partial,” “minimal,” and “incorrect.” 
The percentage of students achieving each rating is shown in Table 4.23, which follows the 
presentation of the sample responses. 
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Only responses that gave the correct ordering of the volumes of the three pyramids 
along with some supporting evidence received a rating of “extended.” In addition, if the student 
did not indicate that the answer was only dependent on height, the numerical values of all three 
volumes were required to be correct. Only five percent of the students participating in the 
Advanced Study reached this level of performance. The following response is an example of an 
“extended” response. 
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Responses that failed to order the three pyramids correctly but that provided correct 
solutions for the volumes of two or three pyramids, or for the heights of Q and R, or that 
correctly compared the heights of Q, P, and R, were rated “satisfactory.” The following response 
was rated “satisfactory” because the volumes of pyramids P and R are correct. 

66 smMsim^tt€Dff , y 99 v°<s§p®'m8® 
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Responses were rated “partial” if they showed a correct solution for either the height or 
the volume of pyramid Q or R. The following response was rated “partial” because 288.7 is the 
volume of pyramid R. 
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Responses that showed or stated that the three pyramids had the same base area or that 
| (area of the base) was the same for all three pyramids were rated “minimal.” Responses that 
contained a correct solution for the volume of pyramid P also received a “minimal” rating. The 
following response was rated “minimal,” since it clearly states that the bases of the three 
pyramids are equal. The area of the bases, however, should have been shown as 100, not 10. 

66 dmnmnmmM" tr<&§pdDm§(B 




Responses that attempted to answer the question, but did not satisfy the conditions for 
at least a “minimal” response, were rated “incorrect.” 
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Performance data on the square pyramid question are presented in Table 4.23. 

This question was very challenging for students because it involved a significant number of 
steps. Students showed a number of approaches to solving the problem. Most students whose 
responses were rated at least “satisfactory” approached the problem analytically, similar to the 
sample responses shown previously. The fact that only 10 percent of the students received 
“satisfactory” or above is somewhat disappointing but may be explained by the fact that many 
students might not have worked with geometric volume problems since they took geometry, 
possibly as long as 2 or 3 years before the assessment. 
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Extended 


Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 
















All Students 


5 


5 


5 


1 1 


52 


20 


Males 


6 


6 


5 


12 


46 


22 


Females 


4 


3 


5 


9 


59 


18 


White 


5 


5 


5 


12 


52 


18 


Black 


0! 


1! 


1 


6 


58 


32 


Hispanic 


3! 


2! 


1! 


5! 


57! 


30! 


Asian/Pacific Islander 


8 


6 


5 


9 


51 


18 


American Indian 


★ * * 


* * * 


* * * 


* * * 


* * * 


* * * 


Are Students Presently 
Enrolled in Mathematics... 














Yes 


5 


5 


5 


1 1 


53 


19 


No 


2! 


2! 


1! 


9! 


46! 


35! 


Are Students Presently 
Enrolled in or Have 
Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 














Yes 


8 


8 


6 


13 


47 


16 


No 


2 


3 


4 


9 


56 


22 


1 















NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The third sample question from the Grade 12 Advanced Study is a multiple-choice 
question assessing students’ ability to select the vector that correctly represented the result of a 
ship’s movement from the origin of a coordinate grid to a position described in terms of the sum 
of two direction vectors. Students had to recognize that the resultant sum, or position, of a trip 
due south for 40 miles followed by a trip of 30 miles southwest could be represented by the 
vector associated with Option D. Sixty-three percent of the grade 12 students correctly selected 
Option D. 
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The only other option that was selected by more than five percent of the students was 
Option E. Twenty-seven percent of the students made this incorrect choice. These students 
selected a vector representing the sum of movement due south and due west rather than the 
sum of movement due south and southwest. 

The problem could be solved in a variety of ways. Most students would not have had 
this type of question (representing and solving problems using vector methods) as part of their 
geometry classroom experience, but those who had studied physics or pre-calculus would have 
studied vectors. The result of 63 percent correct, shown in Table 4.24, is a measure of students’ 
problem-solving skills related to interpreting a situation, representing it, and solving it. 
However, fewer students might have been successful if the task had been to draw the vector 
rather than simply to recognize it, or if they had been required to discriminate between two 
vectors with the same direction but different lengths. 
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Percentage Correct 






All Students 


63 


Males 


67 


Females 


58 


White 


66 


Black 


51 


Hispanic 


50 


Asian/Pacific Islanders 


55 


American Indian 


* ★ ★ 


Are Students Presently 
Enrolled in Mathematics... 




Yes 


63 


No 


52 


Are Students Presently Enrolled in or 
Have Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 




Yes 


67 


No 


60 


i 


-s r.-'-r -- * ^ ^ ■ 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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The final sample question from the Grade 12 Advanced Study was classified as an 
Algebra and Functions question. It presented students with a written description of the motion 
of a Ferris wheel. Using the given information, students were to draw a graph depicting the 
height, over 45 seconds of time, of an individual on a wheel that rotates once every 15 seconds, 
if the individual is at the bottom of the wheel at time 0. In order to answer the question 
correctly, students needed to attend to three conditions: 

O at time t = 0, h = 5; 

O the height can only vary from 5 feet to 35 feet throughout the graph; and 

O the period in the solution graph is 15 seconds (i.e., the graph repeats itself every 
15 seconds). 



This question requires you to show your work and explain your 
reasoning. You may use drawings, words, and numbers in your 
explanation. Your answer should be clear enough so that another person 
could read it and understand your thinking. It is important that you show 
aU of your work. 




10. The Ferris wheel above is 30 feet in diameter and 5 feet above the 
ground. It turns at a steady rate of one revolution each 30 seconds. 
The graph below shows a person's distance from the ground as a 
function of time if the person is at the top of the Ferris wheel at 
time 0. 

On the same graph, draw a second curve that shows a person's 
distance from the ground, as a function of time, if that person is at 
the bottom of the Ferris wheel at time.O and if the Ferris wheel turns 
at a steady rate of one revolution each 15 seconds. Sketch the graph 
from time equals 0 to time equals 45 seconds. 



y 
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Responses were rated “extended,” “satisfactory,” “partial,” “minimal,” or “incorrect.” 
The rating of “extended” was awarded to responses that showed a complete and correct graph 
extending over the 45-second period, as shown below. 

Sao mjpU<s 66 <Bsstt<Bm(M<B(M 99 trces^pczms® 
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Responses containing a graph of three of the conditions but that failed to extend the 
graph from time 0 to 45 seconds were rated “satisfactory.” The sample “satisfactory” response 
shown satisfied three conditions, but the sketch was not shown for the entire time from 
t = 0 to t = 45 seconds. Very few students received “satisfactory” scores. In the sample 
response shown, the response error suggests inattention to the full requirements of the problem 
rather than a lack of understanding. 



y 
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Ratings of “partial” were given to responses showing a graph of two of the conditions. 
The sample “partial” response shown satisfied the conditions that at time t = 0, h = 5 and that 
the height vary from 5 feet to 35 feet throughout the graph, but did not portray the correct 
period of 15 seconds per rotation. 
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“Minimal” ratings were awarded to responses that correctly depicted a graph of one of the 
conditions. The sample “minimal” response shown has the 15-second period graphed correctly. 

SaampU(B 66 mumumaaU 99 d°<Bspdt>ms<s 
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Responses that did not move beyond a simple restatement of the problem were 
rated “incorrect.” 
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Table 4.25 presents student performance data on the Ferris wheel problem. The data 
show that students did relatively well on this question; 39 percent of the responses were rated 
as “extended.” The percentage of students receiving the “extended” rating is somewhat 
surprising, considering student performance on the other problems. This higher level of 
performance may be a reflection of how recently students in the Grade 12 Advanced Study had 
dealt with similar content. The height and vertical variation in the graph indicate the amplitude 
of a periodic function. The compression of the graph horizontally represents the period of the 
graph. Both of these topics, amplitude and periodicity, receive a great deal of attention in the 
study of trigonometric, or circular, functions in pre-calculus and calculus. 
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Extended 


Satisfactory 


Partial 


(Minimal 


Incorrect 


Omit 
















All Students 


39 


0 


18 


13 


19 


10 


Males 


44 


1 


18 


1 1 


14 


10 


Females 


33 


0! 


18 


16 


23 


9 


White 


44 


0 


19 


13 


16 


7 


Black 


13 


0! 


13 


12 


34 


25 


Hispanic 


22! 


0! 


12! 


12! 


30! 


20! 


Asian/Pacific Islander 


33 


0! 


19 


14 


22 


1 1 


American Indian 


★ ★ ★ 


★ ★ ★ 


★ * * 


★ * * 


★ * * 


* * * 


Are Students Presently 
Enrolled in Mathematics... 














Yes 


40 


0 


18 


13 


18 


9 


No 


24! 


0! 


19! 


17! 


21! 


14! 


Are Students Presently 
Enrolled in or Have 
Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 














Yes 


50 


0! 


17 


12 


14 


8 


No 

r 


32 


0 


20 


14 


22 


10 



NOTE: Row percentages may not total 100 due to responses rated "Off Task" or to rounding, or both. 

*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 




ms 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



TTIh(B <n>H dn^aadUcB JL2 aamdl 

Adlwmm^cBdl IPUaacEcBmcBmtt 

An analysis of the data related to the performance of students in the Advanced Study who 
reported that they were currently taking mathematics or who were, or had been, enrolled in 
an AP mathematics course shows that in each case these students outperformed students in the 
study who were not currently taking a mathematics course or who had not taken such an 
AP course. Some of those not currently taking a course may have taken mathematics during the 
first semester of their senior year as part of a block scheduling arrangement at their schools. 
Others may simply have completed their schools’ mathematics curriculum earlier and were 
“lying fallow” mathematically during the assessment semester. Finally, it also is possible that 
some of the students who, by self-report, were not currently enrolled in mathematics had been 
misclassified by the individuals responsible for the NAEP samples at their schools and were 
not really eligible for the Advanced Study. 



Smmmaawn[] 

The background and demographic data collected along with the achievement data indicate 
that both grade 8 and grade 12 students participating in the Advanced Study were different 
from those who did not qualify for the study. They tended to come from homes providing a 
stronger educational context, both in materials and in parental education. Further, these 
students appeared to have somewhat better financial means, as a smaller proportion qualified 
for the federal Free/Reduced-Price Lunch program. Similarly, fewer of them participated in 
Title I programs. 

The Advanced Study was designed to provide students who were taking or had taken 
advanced courses in mathematics with an opportunity to demonstrate their full mathematical 
proficiency. Students at both grade levels who met the criterion for inclusion in the Advanced 
Study performed substantially better than other students on the main NAEP mathematics 
assessment. However, student performance on the Advanced Study itself shows that the study 
questions were quite difficult, even for students who were taking the more challenging 
mathematics courses that were prerequisite to qualify. Overall performance, measured by 
percentage correct, was 36 percent at grade 8 and 30 percent at grade 12, and, at both grade 
levels, most of these students were unable to solve problems that required two or three 
successive steps to achieve the desired result. 
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The 1996 assessment was the first update of the NAEP mathematics assessment framework since 
the release of the National Council of Teachers of Mathematics (NCTM^) Curriculum and 
Evaluation Standards for School Mathematics .* This update reflected refinements in the 
specifications governing the development of the NAEP 1996 mathematics assessment while 
ensuring comparability of results across the 1990, 1992, and 1996 assessments. 
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In addition to the main NAEP 1996 mathematics assessment, three special studies were 
conducted: an Estimation Study, a Theme Study, and an Advanced Study. In each of the studies, 
students were presented with an assessment booklet consisting of blocks of cognitive questions. 
The Estimation and Advanced Studies had a booklet for each grade level, while the Theme Study 
had two different booklets at each grade level. 

The booklets consisted of blocks of cognitive questions and blocks of background 
questions. The cognitive block structure of the special study booklets is displayed in Table A.l. 
The Estimation and Theme booklets began with block M4, a Balanced Incomplete Block (BIB) 
linking block taken from the main assessment. In the Estimation booklets, the linking block 
was followed by two Estimation blocks: a trend block, M16, and a new block, M17. In the 
Theme booklets, the linking block was followed by a single Theme block, either M21 or M22. 

The Advanced booklet began with a nonadvanced block, M20, which was composed of questions 
from various BIB (main assessment) blocks; this was followed by two Advanced blocks, M18 and 
M19. In addition to the cognitive blocks, all of the special study booklets had three blocks in 
common with the main assessment: a general student background block, a mathematics 
background block, and a motivation block. 



National Council of Teachers of Mathematics. (1989| Curriculum and evaluation standards for school mathematics. Reston, 
VA: Author. 
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G®slM)sO 


SsgooSufe© Eflsxifas 


Estimation 

Theme 

Theme 

Advanced Mathematics* 


127 

128 

129 

130 


M4, Ml 6, Ml 7 
M4, M21 
M4, M22 
M20, Ml 8, Ml 9 



Grades 8 and 1 2 only 



The cognitive portion of the Estimation booklet was administered in two sections: the first 
(BIB) block was self-paced; and the two Estimation blocks were administered by a paced audio 
tape. The Theme and Advanced booklets were entirely self-paced. 

The special studies were not part of the main assessment. However, the Estimation and 
Theme samples were drawn from the same population as the main assessment; that is, they were 
chosen to be representative of all students at the target grade level. The grades 8 and 12 
Advanced samples, by contrast, were drawn from populations of students who were considered 
high mathematics achievers based on their enrollment in advanced mathematics classes during 
the 1995—1996 school year. 
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The populations for the special studies were sampled as part of the complex multistage sample 
design used for the main national sample. The design involved sampling students from selected 
schools within 94 selected geographic areas, called primary sampling units (PSUs), across the 
United States. There were five steps in the selection process: 

1. Selection of geographic PSUs (counties or groups of counties). 

2. Selection of schools within PSUs. 

3. Assignment of sample types to schools. 

4. Assignment of session types to schools. 

5. Selection of students for session types within schools. 

The samples for the main and special studies were drawn for grades 4, 8, and 12. 

In addition, separate age-based samples were selected for the long-term trend studies in 
mathematics. Table A. 2 shows the numbers of students and schools that were assigned to each 
condition. In addition to representing the respective populations as a whole, the main and special 
study samples involved oversampling of nonpublic schools, and of public schools with moderate 
or high enrollment of Black or Hispanic students. This oversampling was undertaken to increase 
the sample sizes of nonpublic school students and minority students, so as to increase the 
reliability of estimates for those groups of students. The assessment period for the main 
assessment and special studies was in the winter of 1996, between January 3 and 
March 29. 
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Number of 
Assessed Students 


Number of 
Schools 


Wlean Number of 
Students Per 
Assessment 
Per School 


ftflean Number of 
Students Per Item 
Per School 












Age Class 9 










Long-Term Trend 










Print Booklets 51-56 


5,019 


215 


23.3 


3.9-7. 8* 


Tape Booklet 91 


1,852 


127 


14.6 


14.6 


Tape Booklet 92 


1,721 


1 16 


14.8 


14.8 


Tape Booklet 93 


1,840 


125 


14.7 


14.7 


Grade 4 Main 










Print Mathematics 


10,830 


445 


24.3 


5.6 


Print Science 


11,578 


421 


27.5 


4. 5-7.4* 


Tape Mathematics Estimation 


2,1 15 


120 


17.6 


17.6 


Print Mathematics Theme 


4,004 


230 


17.4 


8.7 


Age Class 1 3 










Long-Term Trend 










Print Booklets 51-56 


5,493 


221 


24.9 


4. 1-8. 3* 


Tape Booklet 91 


1,928 


128 


15.1 


15.1 


Tape Booklet 92 


1,866 


125 


14.9 


14.9 


Tape Booklet 93 


1,864 


124 


15.0 


15.0 


Grade 8 Main 










Print Mathematics 


1 1,521 


411 


28.0 


6.5 


Print Science 


1 1,971 


346 


34.6 


5. 6-9. 4* 


Tape Mathematics Estimation 


2,244 


104 


21.6 


21.6 


Print Mathematics Theme 


4,227 


175 


24.2 


12.1 


Print Advanced Mathematics 


2,365 


253 


9.3 


9.3 


Age Class 1 7 










Long-Term Trend 










Print Booklets 51— 56 


4,669 


186 


25.1 


4. 2-8. 4* 


Tape Booklet 84 


1,848 


133 


13.9 


13.9 


Tape Booklet 85 


1,691 


122 


13.9 


13.9 


Grade 1 2 Main 










Print Mathematics 


10,600 


430 


24.8 


5.7 


Print Science 


1 1,481 


401 


28.6 


4. 6-7. 7* 


Tape Mathematics Estimation 


1,889 


96 


19.7 


19.7 


Print Mathematics Theme 


3,860 


196 


19.7 


9.8 


Print Advanced Mathematics 


2,965 


207 


14.3 


14.3 


Print Advanced Science 


2,431 


222 


11.0 


11.0 


i 



* This number varied because some item blocks appeared more often than others in the set of booklets used for this sample. 
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Following is a brief summary of the general features of the sampling procedure. 

Many details and adjustments left out of this description can be found in Chapter 3 of The NAEP 
1996 Technical Report . 2 

Of the population of 1,000 PSUs in the country, the 22 largest PSUs were included in the sample 
with certainty. The remaining PSUs were partitioned into 72 strata (defined by region and various 
socioeconomic characteristics), and one PSU was chosen from each stratum with probability 
proportional to size (of the population of the PSU). 

QDff S<e/fo(H>(H>5s 

In the second-stage sampling a list was created, for each grade, of all public and nonpublic 
schools in the 94 selected PSUs. Schools were selected (without replacement) across all PSUs 
with probabilities proportional to assigned measures of size. Details of the assignment of school 
size can be found in Section 3.3 of The NAEP 1996 Technical Report. 

snap mm® mil qdU SmmfpUcB TTdujpcB Jt<n> S<£Zfoaz>az)0s 

In order to determine the effect of using different guidelines for excluding or assessing students 
with disabilities or limited English proficiency, three different sample types were assigned to 
schools selected for the main or special study assessments. These sample types were: 

O Si, in which students were subject to the exclusion criteria used in 1990 and 1992; 

O S2, in which students were subject to new 1996 exclusion criteria; and 

O S3, in which students were subject to new 1996 exclusion criteria and 
accommodations were offered to students with disabilities (SD) and limited 
English-proficient (LEP) students. 3 



2 Allen, N. L., Carlson, J. E., & Zelenak, C. A. (1999). The NAEP 1996 technical report . Washington, DC: National Center for 
Education Statistics. 

1 See Chapter 5 of the NAEP 1996 Technical Report for a fuller description of sample types, exclusion criteria, 
and accommodations. 
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Sample type was assigned to schools separately for each grade. For schools that were not 
also selected for the State Assessment, sample type was assigned to schools according to the 
percentages shown in Table A. 3. 
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in the 1996 Assessment 




s 







SI 


S2 


S3 










4 


20 


45 


35 


8 


16 


42 


42 


12 


15 


46 


39 



c 
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The first step in sampling students was to compile a list of all grade-eligible students in a school. 
Along with this list, schools at grades 8 and 12 were asked to provide course enrollment 
information to be used in determining eligibility for the Advanced Study. Specifically, eligibility 
was based on reported enrollment in Algebra 1 or beyond at grade 8 and on reported enrollment 
in Algebra 3, Pre-Calculus, Calculus and Analytical Geometry, Calculus, or AP Calculus at 
grade 12. 

Systematic selection of students from the grade-eligible list was made to obtain the target 
sample size for each school. Westat, Inc. district supervisors then assigned sampled students to 
one of the various session types according to specified procedures. Session types included the 
various subject areas and, within the subject area of mathematics, whether the students would be 
included in the main (BIB), long-term trend, Estimation, Theme, or Advanced assessments. 
Obviously, only students who were eligible for the Advanced Study could be assigned to Advanced 
sessions, but students eligible for the Advanced Study also were assigned proportionally to all 
other session types. 

To ensure an adequate sample size of SD/LEP students, oversampling procedures were 
applied to these students in sample type 2 (S2) and sample type 3 (S3). In general, SD/LEP 
students were sampled (within schools) at twice the rate at which non-SD/LEP students were 
sampled. For grades 8 and 12, because of the way sessions were assigned, oversampling 
of SD/LEP students took place only among students who were not eligible for the 
Advanced assessment. 
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The exclusion and accommodation conditions that were applied in S3 are anticipated to be 
similar to those that will be used in future NAEP assessments. Consequently, the data from S3 
constitute a statistical bridge that can be used in maintaining future trend lines. However, in order 
to allow timely reporting of 1996 results, data from S3 were generally not included in the 
reporting sample, but were set aside for further analysis of the impact of the availability of 
accommodations on group achievement estimates. 

The following tables — A. 4 through A.6 — illustrate how the special studies were 
distributed across the Si, S2, and S3 samples. In the tables, the notation “B” refers to that subset 
of the participating students who were identified as having either disabilities or limited English 
proficiency. “A” refers to all other participating students. As can be seen, the data reported for the 
Estimation Study were generally drawn from Si, but S2 students were added at grade 12 in order 
to obtain a sufficient sample size. However, to insure comparability across grades, results from S2 
students with disabilities or limited English proficiency were not reported. The Theme Study 
sample was drawn from S2. The Estimation and Theme Studies also were administered to S3, as 
a statistical bridge to future administrations of these studies. The Advanced Study sample was 
taken from S3 at grade 12 and, in order to obtain a sufficient sample size, from both S2 and S3 at 
grade 8. Due to the nature of the Advanced Study selection criteria, accommodations were 
infrequent in this sample, particularly at grade 12. Consequently, it was judged acceptable to 
report Advanced Study results from S3. As a precaution, however, data from grade 8 students 
with disabilities or limited English proficiency (where some accommodations did occur) were held 
back from reporting. 
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S3 


R 


X 
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NR 
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R 


R 
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NR 


NR 



An "R" indicates that this cell was part of the reporting sample. An "NR" cell indicates that data were collected but not 
included in the reporting sample. An "X" indicates that no data were collected from that category. 
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* An "R" indicates that this cell was part of the reporting sample. An "NR" cell indicates that data were collected but not 
included in the reporting sample. An "X" indicates that no data were collected from that category. 
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included in the reporting sample. An "X" indicates that no data were collected from that category. 
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In order for the data to be reported, NCES school and student participation rates must be met. 
School nonparticipation or student nonparticipation in the form of absenteeism creates a potential 
for bias to be introduced in the reporting of the data. The participation rates of schools and students 
included in the 1996 assessments were inspected for any differences. NCES standards regarding 
acceptable potentials for bias are expressed in terms of weighted participation rates. Table A. 7 
shows the weighted participation rates by grade and session type for the main reporting samples. 

For the main samples, the student participation rates are similar for different session types at grades 
4 and 8, but the student participation rates at grade 12 and the school participation rates at all 
grades vary by session type. The differential school participation rates show that different session 
types include different schools. This is due to the assignment of schools to sample type, the fact that 
all session types were not assessed in all sample types, and the specific sample types included in 
the reporting populations for each session type. 
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School Participation 


82.3 


77.8 


93.5 


77.9 


— 


— 


Student Participation 


95.3 
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96.7 


95.4 


— 


— 


Overall Participation 


78.4 
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90.4 
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School Participation 


81.5 


79.7 


85.3 


86.8 


77.0 


— 


Student Participation 


92.9 


93.1 


93.8 


92.7 


95.6 


— 


Overall Participation 


75.7 


74.3 


80.0 


80.4 


73.6 


— 


US 1 














School Participation 


76.2 


77.4 


63.9 


78.4 


77.6 


77.7 


Student Participation 


82.3 


77.5 


81.0 


78.2 


85.8 


86.5 


Overall Participation 


62.7 


60.0 


51.7 


61.3 


66.6 


67.2 
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Although school and student nonresponse adjustments are intended to reduce the potential for 
nonparticipation to bias the assessment results, they cannot completely eliminate this potential 
bias with certainty. The extent of bias remains unknown since there are no assessment data for 
the nonparticipating schools and students. 

Some insight can be gained about the potential for residual nonresponse bias by 
examining the weighted school- and student-level distributions of characteristics known for both 
participants and nonparticipants, especially for those characteristics known or thought likely to 
be related to achievement on the assessment. If the distributions for the full sample of schools 
(or students) without the use of nonresponse adjustments are close to those for the participants 
with nonresponse adjustments applied, there is reason to be confident that the bias from 
nonparticipation is small. 

A nonresponse bias analysis completed on the reporting population for the science 
assessment can be found in the NAEP 1996 Technical Report. 4 Science was chosen because it 
contained the largest number of students and could, therefore, provide the most precise 
estimates of student distributions across several demographic characteristics (i.e., age, race, 
gender, type of school and school size). Generally, the findings show that the student distributions 
before and after school and/or student nonresponse adjustments were similar, with a few 
exceptions. Most of these exceptions were at grade 12 due to its relatively high nonresponse rate 
(20.3% for grade 12 students). An additional nonresponse bias analysis was completed on the 

4 Allen, et al., (1999). op. oil. 
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reporting populations of Mathematics, Mathematics Theme, Mathematics Estimation and 
Advanced Mathematics. Each of these reporting populations have smaller numbers of schools 
and students in the sample. Thus, the variance of the estimated distributions will be larger for 
these subjects, than for science. 

Within the NAEP data, there are several school-level characteristics available for both 
participating and nonparticipating schools. The tables that follow show the combined impact of 
nonresponse and of the nonresponse adjustments on the distributions of schools (weighted by the 
estimated number of eligible students enrolled) and students, by the type of school (public, 
Catholic, other nonpublic), the size of the school (measured by the estimated number of eligible 
students enrolled) and whether the school is located in an urban/rural place. Three size classes 
have been defined for each grade. The data are for the 1996 samples excluding the science 
assessment. 

Several student-level characteristics are available for both absent and assessed students. 
The tables that follow show the impact of school nonresponse and nonresponse adjustments, and 
student nonresponse and nonresponse adjustments on the distributions of eligible students for 
each grade. The distributions are presented by age category (at or below modal age, and above 
modal age), race/ethnicity category (White, Black, Hispanic, and other), gender, SD and LEP. 
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Table A. 8 shows the weighted marginal distributions of students for each of the three 
classification variables for grade 12, using weighted eligible schools. The distributions before 
school nonresponse adjustments are based on the full sample of in-scope schools for each 
assessment — those participating, plus those refusals for which no substitute participated. 

The distributions after school nonresponse adjustments are based only on participating schools for 
each assessment, with school nonresponse adjustments applied to them. The weighted 
school-level nonparticipation rates at grade 12 are as follows: Mathematics, 27.8 percent; 
Mathematics Theme, 21.6 percent; Mathematics Estimation, 36.1 percent; and Advanced 
Mathematics, 22.4 percent. For more detail, see the NAEP 1996 Technical Report. 5 




%) ©if P@pula^i©ns ©{HiSIglBie the NATION’S 
TtuSB Weighted Sample @§ Eligible REPORT 
md M ffer Seh&el N@ns"®sp@nse ^ARD 
1996 Main NAEP Samples 
Sdeme}, ©rode 12 





Mathematics 


Mathematics 

Theme 


Mathematics 

Estimation 


Advanced 

Mathematics 


Advanced Science 


IF\spJ(oi(in<sM 


Before 


After 


Before 


After 


Before 


After 


Before 


After 


Before 


After 


School Type 






















Catholic 


5.8 


6.6 


4.5 


5.5 


7.5 


7.1 


4.5 


5.2 


5.4 


5.5 


Other Nonpublic 


3.6 


3.1 


4.5 


3.8 


3.9 


4.8 


4.1 


3.4 


3.8 


3.1 


Public 


90.6 


90.3 


91.0 


90.8 


88.5 


88.1 


91.4 


91.4 


90.7 


91.4 


School Size 1 






















1 (1-49) 


6.0 


6.2 


5.7 


5.1 


6.8 


8.3 


7.1 


7.3 


5.5 


5.0 


2 (50-399) 


67.9 


66.0 


71.6 


71.4 


72.1 


66.4 


72.7 


72.9 


69.4 


65.7 


3 (400+) 


26.1 


27.7 


22.7 


23.5 


21.1 


25.3 


20.2 


19.9 


25.1 


29.2 


School Location 






















Large City 


14.8 


15.1 


14.9 


17.5 


14.7 


15.4 


13.5 


15.1 


15.4 


16.0 


Midsize City 


18.6 


21.2 


14.6 


15.6 


20.8 


23.4 


. 17.3 


18.2 


16.3 


18.4 


Urban Fringe/ 






















Large City 


22.4 


19.1 


27.5 


23.5 


18.5 


19.5 


25.0 


22.6 


23.8 


22.5 


Urban Fringe/ 






















Midsize City 


14.5 


15.2 


13.3 


15.2 


17.5 


14.4 


14.3 


15.2 


15.0 


13.9 


Large Town 


1.1 


1.2 


0.7 


0.9 


0.7 


1.1 


0.5 


0.7 


1.5 


0.9 


Small Town 


12.1 


13.1 


14.5 


14.6 


11.5 


11.5 


16.0 


16.1 


14.7 


15.9 


Rural MSA 


3.8 


3.0 


6.0 


5.7 


4.5 


2.1 


5.6 


4.5 


4.1 


4.1 


Rural nonMSA 


12.8 


12.0 


8.4 


7.0 


11.8 


12.6 


7.7 


7.6 


9.1 


8.4 


i 



'Distributions by school size are not comparable to previous assessments, since students were eligible by grade only (instead 
of by grade or age) in 1996. School size refers to the number of eligible students enrolled. 



5 Allen, et al., (1999). op. cit. 
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It can be seen from Table A.8 that, overall, the distributions for school type, school size 
and school location remain similar. For Mathematics, even though the nonresponse rate is 27.8 
percent, the only exceptions may be midsize cities and urban fringe of large cities; for 
Mathematics Theme, the exceptions may be large cities and their urban fringe; for Mathematics 
Estimation, exceptions may be at medium and large schools, midsize cities and their urban 
fringe, and rural MSAs (Metropolitan Statistical Areas). For Advanced Mathematics, even though 
the school nonparticipation rate is 22.4 percent, the only exception may be urban fringes of large 
cities. The potential for bias is greatest for Mathematics Estimation, which also has the highest 
nonparticipation rate for schools. 

Table A.9 shows the distributions of two school-level characteristics — school type 
and school location, plus additional distributions of student-level characteristics, using 
weighted eligible students. The distributions before student nonresponse adjustments are 
based on assessed and absent students (with base weights adjusted for school 
nonparticipation) .The distributions after student nonresponse adjustments are based on 
assessed students only, with the student nonresponse adjustments also applied to them. The 
rates of student nonparticipation for the five subjects at grade 12 are as follows: Mathematics, 
17.7 percent; Mathematics Theme, 21.8 percent; Mathematics Estimation, 19.0 percent; 
Advanced Mathematics, 14.2 percent. For more detail, see the NAEP 1996 Technical Report . 6 



6 Allen, et al., (1999). op. cit. 
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Mathematics 


(Mathematics 

Theme 


Mathematics 

ktimntmn 


Advanced 

fyinfnomrrtirc 


AflMHn/flN C/lan/a 


f^»)QD()aC 6 ® 0 D 






















Before 


After 


Before 


After 


Before 


After 


Before 


After 


Before 


After 


School Type 






















Catholic 


7.2 


8.0 


5.9 


7.5 


7.9 


9.2 


9.1 


10.6 


10.6 


1 1.7 


Other Nonpublic 


3.3 


3.8 


3.8 


4.1 


5.2 


6.1 


7.1 


7.9 


4.9 


5.4 


Public 


89.5 


88.1 


90.3 


88.3 


86.9 


84.7 


83.7 


81.5 


84.5 


82.9 


School Location 






















Large City 


15.3 


14.6 


17.2 


17.3 


15.0 


14.5 


15.5 


16.9 


15.5 


15.2 


Midsize City 
Urban Fringe/ 


21.3 


20.8 


15.8 


15.4 


23.4 


23.5 


20.4 


19.8 


16.7 


15.6 


Large City 
Urban Fringe/ 


19.2 


18.8 


23.8 


23.6 


19.2 


19.4 


19.5 


18.8 


27.1 


27.6 


Midsize City 


14.4 


15.2 


15.0 


15.7 


14.3 


15.1 


15.5 


14.8 


12.1 


12.9 


Large Town 


1.3 


1.2 


0.9 


0.9 


1.3 


1.2 


0.7 


0.7 


0.8 


0.7 


Small Town 


13.2 


13.1 


14.6 


14.3 


1 1.5 


10.7 


14.4 


14.6 


13.9 


13.5 


Rural MSA 


2.9 


3.1 


5.9 


5.8 


2.0 


2.2 


6.0 


6.6 


5.6 


5.9 


Rural nonMSA 

Age Category 
At Modal Age 


12.5 


13.1 


6.8 


7.0 


13.2 


13.4 


7.9 


7.9 


8.3 


8.7 


or Younger 
Older than 


65.9 


66.0 


65.4 


65.7 


67.0 


66.6 


74.0 ’ 


73.7 


72.5 


72.3 


Modal Age 


34.1 


34.0 


34.6 


34.3 


33.0 


33.4 


26.0 


26.3 


27.5 


27.7 


Race/Ethnicity 






















White 


70.0 


70.0 


69.1 


68.6 


70.5 


70.9 


74.8 


74.1 


74.8 


74.1 


Black 


13.1 


13.4 


13.1 


13.0 


12.7 


12.8 


7.3 


7.2 


8.9 


9.1 


Hispanic 


10.1 


9.6 


10.6 


1 1.2 


1 1.6 


10.9 


7.5 


7.9 


6.8 


7.0 


Other 


6.8 


7.0 


7.1 


7.2 


5.1 


5.4 


10.4 


10.8 


9.5 


9.7 


Gender 1 






















Male 


47.9 


47.6 


49.8 


49.1 


47.4 


47.3 


50.2 


50.3 


49.4 


49.2 


Female 


52.1 


52.4 


50.2 


50.8 


52.6 


52.7 


49.1 


49.0 


50.5 


50.8 


SD 






















Yes 


2.7 


2.7 


2.5 


2.2 


2.7 


2.4 


0.6 


0.7 


0.4 


0.3 


No 


97.3 


97.3 


97.5 


97.8 


97.3 


97.6 


99.4 


99.3 


99.6 


99.7 


LEP 






















Yes 


1.1 


1.1 


1.8 


1.6 


0.9 


1.0 


1.2 


1.2 


1.1 


1.0 


No 


98.9 


98.9 


98.2 


98.4 


99.1 


99.0 


98.8 


98.8 


98.9 


99.0 


SD, LEP 






















SD Yes; LEP Yes 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


SD Yes; LEP No 


2.7 


2.7 


2.4 


2.2 


2.7 


2.4 


0.6 


0.7 


0.4 


0.3 


SD No; LEP Yes 


1.1 


1.1 


1.8 


1.6 


0.9 


1.0 


1.2 


1.2 


1.1 


1.0 


SD No; LEP No 

1 


96.2 


96.2 


95.7 


96.2 


96.3 


96.5 


98.2 


98.1 


98.5 


98.6 

' - ! 



' Gender is unknown for a small percentage of students. 
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Table A. 9 shows that with two exceptions at grade 12, for the distributions of type of 
school attended and place where the school is located, the effect of the student nonparticipation 
adjustment has resulted in very little change in distribution. The changes in the distribution of 
school type for Mathematics Estimation and Advanced Mathematics reflect the relatively high 
nonresponse rate of grade 12 public school students (22.7% versus 8.5% for nonpublic school 
students; from the NAEP 1996 Technical report). 7 

Table A. 10 shows the weighted distributions of eligible students in participating schools, 
using the base weights of assessed and absent students unadjusted for school-level nonresponse. 
Tables A.9 and A. 10 show that both school and student-level nonresponse and nonresponse 
adjustments have little effect on the distributions of eligible students by age, race/ethnicity, 
gender, and SD and LEP status. All of the distributions in the tables are similar. 
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Mathematics 


Mathematics 

Theme 


Mathematics 

[Estimation 


Advanced 

Mathematics 


Advanced Science 


(Jte(pQD0cE)[feXiO 












Age Category 












At Modal Age or Younger 


65.6 


64.9 


66.3 


74.0 


72.1 


Older than Modal Age 


34.4 


35.1 


33.7 


26.0 


27.9 


Race/Ethnicity 












White 


71.0 


69.1 


70.8 


75.4 1 


76.0 


Black 


12.8 


13.4 


12.6 


7.5 


8.8 


Hispanic 


9.5 


10.3 


11.6 


7.3 


6.3 


Other 


6.8 


7.1 


5.1 


9.8 


9.0 


Gender 1 












Male 


47.9 


49.9 


47.8 


49.8 


49.6 


Female 


52.1 


50.0 


52.2 


49.4 


50.3 


SD 












Yes 


2.6 


2.5 


2.8 


0.5 


0.4 


No 


97.4 


97.5 


97.2 


99.5 


99.6 


LEP 












Yes 


1.0 


1.8 


1.0 


1.1 


1.0 


No 


99.0 


98.2 


99.0 


98.9 


99.0 


SD, LEP 












SD Yes; LEP Yes 


0.0 


0.0 


0.0 


0.0 


0.0 


SD Yes; LEP No 


2.6 


2.5 


2.8 


0.5 


0.4 


SD No; LEP Yes 


1.0 


1.8 


1.0 


1.1 


1.0 


SD No; LEP No 
1 — 


96.3 


95.7 


96.2 


98.4 


98.6 

■ 



Gender is unknown for a small percentage of students. 



Allen, et al., (1999). op. cit. 
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When comparing the distributions in Table A. 9 before and after student nonresponse 
adjustments, we expect the distributions by age category and race/ethnicity to be similar because 
these variables were used to determine student nonresponse adjustment classes. However, the 
distributions by gender and SD and LEP status are also similar. To the extent that nonrespondents 
would perform like respondents with the same characteristics (defined by the characteristics in 
the tables), the bias in the assessment data is small. 

Further information about potential nonresponse bias can be gained by studying the 
absent students. NAEP proficiency estimates are biased to the extent that assessed and absent 
students within the same weighting class differ in their distribution of proficiency. It seems likely 
that the assumption that absent students are similar in proficiency to assessed students is 
reasonable for some absent students — namely, those whose absence can be characterized as 
random. Conversely, it seems likely that students with longer and more consistent patterns of 
absenteeism — such as truants, dropouts, near dropouts, and the chronically ill — are unlikely to 
be as proficient as their assessed counterparts. 

In the 1996 assessments, schools were asked to classify each absent student into one of 
nine categories. The results of this classification for the assessments are shown in TableA.il. 
Table A. 11 shows that, as anticipated, the majority of absenteeism from the assessment was the 
result of an absence from school of a temporary and unscheduled nature. The table shows that 
among the two Advanced sessions, the absenteeism rate is lower than among the ‘non-Advanced’ 
sessions. The proportion of absenteeism classified as temporary is similar across subjects, 
including science (63.6%). 
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(Mathematics 


Mathematics 

Theme 


(Mathematics 

Estimation 


Advanced 

(Mathematics 


Advanced Science 


GtoGQD0©®(? 












Temporay Absence 1 


65.2 


59.9 


58.9 


60.9 


64.5 


Long-term Absence 2 


1.3 


1.0 


2.0 


1.0 


0.0 


Chronic Truant 


1.0 


1.4 


1.5 


0.1 


0.0 


Suspended or Expelled 


0.4 


0.2 


0.3 


0.1 


0.3 


Parent Refusal 


8.1 


5.8 


12.8 


1 1.1 


1 1.7 


Student Refusal 


10.8 


14.7 


6.9 


14.3 


12.9 


In School, Did not 












Attend Session 


8.2 


10.6 


9.3 


9.3 


5.7 


In School, not Invited 3 


0.0 


0.0 


0.0 


0.0 


0.0 


Other 


4.9 


6.3 


8.3 


3.2 


4.8 


Missing 


0.0 


0.0 


0.0 


0.0 


0.0 


Total Absentee Sample 


2,598 


1,112 


449 


460 


379 


Total Sample Size 


13, 258 


4,972 


2,338 


3,425 


2,810 


Overall Absentee Rate 


19.6 


22.4 


19.2 


13.4 


13.5 


1 



1 Absent less than two weeks due to illness, disability, or excused absence. 

2 Absent more than two weeks due to illness or disability. 

3 In school, but not invited to assessment session due to disruptive behavior. 



For each subject in grade 12, a significant component of absenteeism is not temporary or 
due to parental refusal. Chronic truants, those suspended, and those in school but not attending, 
constitute the obvious candidates for potential bias. These groups comprise 6.0 to 9.5 percent of 
absent students in the Advanced sessions (or 0.8% to 1.3% of each total sample). Among the 
non-Advanced sessions, the groups comprise 9.6 to 12.2 percent of absent students (or 1.8% to 
2.7% of each total sample). Thus their potential for introducing significant bias under the current 
procedures is minor. 

As with all NAEP assessments, data collection was conducted by trained Westat field 
staff. Materials collected as part of the 1996 assessment were shipped to National Computer 
Systems, where trained staff evaluated the responses to the constructed-response questions using 
scoring rubrics or guides prepared by the Educational Testing Service (ETS). 

Each constructed-response question had a unique scoring rubric that defined the 
criteria used to evaluate students’ responses. The extended constructed-response questions 
were evaluated with four- or five-level rubrics (e.g., no evidence of understanding, evidence of 
minimal understanding, evidence of partial understanding, and evidence of satisfactory or 
extended understanding). Short constructed-response questions first appearing in the 1996 
assessment were rated according to three-level "rubrics that permitted partial credit (e.g., evidence 
of little or no understanding, evidence of partial understanding, and evidence of full 
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understanding). Other short constructed-response questions that were earned over from previous 
assessments were scored as either “correct” or “incorrect.” For more information, see the NAEP 
1996 Technical Report . 8 

Student responses for constructed responses also were scored as “off task” if the student 
provided a response that was deemed not related in content to the question asked. A simple 
example of this type of response is, “I don’t like this test.” In contrast, responses scored as 
“incorrect” were valid attempts to answer the question that were simply wrong. 



AhmaaU'Ojjsns IPv°(iD(i?(BdJwv°(BS 

The results from the Estimation Study were analyzed and reported in terms of the NAEP 
proficiency scale metric, while the results from the Theme Study and the Advanced Study were 
reported simply in terms of block percent correct scores. It was possible to scale the results from 
the Estimation Study because all of the questions were presumed to measure a relatively 
unidimensional mathematical trait: the ability to estimate and work with estimated data. However, 
no such single trait was hypothesized to underlie the questions from either the Theme Study or 
the Advanced Study. Rather, Theme Study questions were characterized by being presented in a 
single practical context, and Advanced Study questions were characterized by their more 
challenging content, some of which might only be covered in advanced courses. 

For the Theme and Advanced Studies, another alternative might have been to scale the 
items onto the mathematics scales used in the main NAEP assessment. However, this would have 
required a very large number of linking questions because the main reporting scale in 
mathematics is actually a composite of five separately scaled subscales representing the various 
mathematics content areas. Theme Study and Advanced Study questions therefore would have 
had to be disaggregated onto five subscales for analysis and scaling, and stable links would have 
been required for each subscale. 

JEsMmmUwm 

The Estimation booklet at each grade was scaled with an IRT analysis, and proficiencies were 
calculated for students and put on the scale of the 1992 Mathematics Estimation assessment. The 
calculation of proficiencies for each grade involved a number of steps. First, the Estimation data 
were scaled, using the PARSCALE program to estimate item parameters. This scaling was 
entirely separate from the analysis of the main RIR data, and data from the non-estimation block, 
M4, were not used. 

Next, conditioned proficiency scores were created. Here, the same background variables 
and the same resulting contrasts as in the main assessment were used. However, because the 
principal components of the contrasts are sample dependent, these principal components were 
calculated separately for the Estimation and the main data. 



5 Allen, et al., (1999). op. cit. 
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Conditioned scores are initially in an arbitrary metric, with a mean approximately zero and a 
standard deviation approximately one. Therefore, the conditioned Estimation scores were linearly 
transformed in order to place them onto the 1992 Estimation proficiency scale. That is, each score was 
multiplied by one constant and then another constant was added to it, in the form: 

P = A*x + B, 

t i 7 

where x. is the score for student i in the arbitrary metric, P. is the score for student i in the 
1992 Estimation scale metric, and A and B are appropriate constants. These constants, called 
transformation constants, also are sample dependent and therefore specific to the 
Estimation samples. 

Table A. 12 shows the A and B constants used to transform the initial 1996 Estimation scores 
from an arbitrary metric to the 1992 metric. 
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A 


B 


@ 00^3 






Grade 4 


37.9729 


207.2608 


Grade 8 


28.0702 


269.9908 


Grade 1 2 


27.9881 


295.8688 






For this report, the following Estimation Study performance data were presented for each 
grade and for selected subgroups within each grade: 

O average proficiency means, with standard errors reflecting sampling and 
measurement error; 

o average proficiency means at percentile levels — 5, 10, 25, 50, 75, 90, and 95; and 

O the percentage of students at or above achievement levels of Advanced, Proficient , 
Basic , and below Basic. 
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Block Percent Correct Scores for Each Student. The linking block, M4, was dropped 
from the analysis, and a block percent correct score for each student was calculated using the 
Theme block (M21 or M22). The block percent correct score was calculated as a straight 
percentage correct with not-reached items counted as wrong. Dichotomous items were scored 0 
(wrong) and 1 (correct), and polytomous items were scored from 0 to m - 1 (m being the number 
of categories in a given polytomous item), so 0 = 0, the first partial credit category = 1, the 
second = 2 ,..., and so on to m - 1. 

The student percent correct scores were broken down by: 

O gender; 

O race/ethnicity; 

O frequency of writing a few sentences about how to solve a mathematics problem; and 
O frequency of writing reports or doing a mathematics project. 

For the “frequency of writing” variables, teacher responses were used at grades 4 and 8. 
Because teachers of twelfth-grade students were not surveyed, student responses were used at 
grade 12. 

Significance tests were run for families of tests defined by the contrasts among levels of 
each variable. This was accomplished by using standard almanac programs. 

Item-hy-Item Statistics for Questions in the Released Block. For the individual item 
statistics for questions in the released Theme blocks at each grade, the following procedures 
were used: 

O For dichotomous items, percentages responding to each alternative were 

calculated, but generally only the percentages for the correct response were included in 
the report. 

O For polytomous items, percentages responding to each partial credit category were 
calculated and generally reported. 

O The percentages responding in categories were reported for the whole group and were 
also broken down by categories of four variables: 

1. gender; 

2. race/ethnicity; 

3. frequency of writing a few sentences about how to solve a mathematics 
problem; and 

4. frequency of writing reports or doing a mathematics project. 
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Booklet percent correct scores were calculated for each student in a manner similar to the block 
percents created for the Theme analysis. The difference is that the two Advanced blocks were 
analyzed together to create a single booklet percent correct score for each student. The linking 
block, M20, was deleted from the analysis. 

The booklet percent correct scores were reported for the whole group and for subgroups 
defined by gender and race/ethnicity. In addition, for grade 12, a breakdown of the booklet 
percents was done by two self-report variables: (1) whether the student was currently taking 
mathematics and (2) whether the student was taking or had taken an AP mathematics course. 

Response category frequencies and percentages were reported on various background, 
course-taking, and classroom practices variables. These responses also were reported 
for a comparison sample of students from the main assessment who were not eligible for the 
Advanced Study. 

For each group, information on the following demographic variables was reported: 

O gender; 

O race/ethnicity; 

O parent education; 

O home resources; 

O region; 

O location; 

O type of school; 

O Title 1 participation; and 

O eligibility for the federal Free/Reduced-Price School Lunch program. 

In addition, information was reported on the following classroom practices variables: 

O emphasis on mathematics topics; 

O emphasis on mathematics skills; 

O unrestricted use of calculators; 

O permitted use of calculators on tests; 

O access to calculators; 

O provision of instruction in use of calculators; 
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O availability of graphing calculators; and 
O primary use of graphing calculators. 

The classroom practices variables are based on teacher responses. At grade 12, these were 
only available for the Advanced sample because grade 12 teachers in the main sample were 
not surveyed. 

In addition, the report provides information on the average proficiency of the 
Advanced-eligible and not-Advanced-eligible students in the main sample. Because the 
Advanced-eligible students in the main sample were selected by exactly the same criterion as the 
students who actually participated in the Advanced Study, this affords a direct comparison of the 
mathematics achievement of the two groups. 

Item-by-item statistics for items in both Advanced blocks were reported for grades 
8 and 12. 

a) For dichotomous items this included percentages responding to each alternative, with 
the key (correct) alternative marked. 

b) For polytomous items this included percentages responding to each partial 
credit category. 

c) The percentages responding in categories were shown for the whole group and broken 
down by the variables: 

O gender, and 
O race/ethnicity. 

d) In addition for grade 12, the breakdown variables also included: 

O whether the student was currently taking mathematics; and 
O whether student had taken an AP course in mathematics. 

RJA1IEJP (B^dDmjps 

In this report, results are provided for groups of students defined by shared characteristics — 
gender, race/ethnicity, parental education, type of school, region of the country, participation in 
Title I programs, and eligibility for the federal Free/Reduced-Price School Lunch program. 
Results are reported for subpopulations only when sufficient numbers of students and adequate 
school representation are present. For public and nonpublic school students in the national 
assessment, the minimum requirement is at least 62 students in a particular subgroup from at 
least 5 primary sampling units (PSUs). 5 However, the data for all students, regardless of whether 
their subgroup was reported separately, were included in computing overall results. Definitions of 
the subgroups referred to in this report are presented below. 



For the national assessment, a PSU is a selected geographic region (a county, group of counties, or metropolitan statistical area). 
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Results are reported separately for males and females. 

I Susa?©/© ttBamneMnu 

The race/ethnicity variable is derived from school records and two questions asked of students 
from the set of general student background questions. 



If you are Hispanic, what is your Hispanic background? 
O I am not Hispanic 

O Mexican, Mexican American, or Chicano 
O Puerto Rican 
O Cuban 

O Other Spanish or Hispanic background 



Students who responded to this question by filling in the second, third, fourth, or fifth oval 
were considered Hispanic. For students who filled in the first oval, did not respond to the 
question, or provided information that was illegible or could not be classified, responses to the 
following question were examined to determine their race/ethnicity. 



Which best describes you? 

O White (not Hispanic) 

O Black (not Hispanic) 

O Hispanic (“Hispanic” means someone who is Mexican, Mexican 
American, Chicano, Puerto Rican, Cuban, or of other Spanish or 
Hispanic background.) 

O Asian or Pacific Islander (“Asian or Pacific Islander” means someone 
who is from a Chinese, Japanese, Korean, Filipino, Vietnamese, or 
other Asian or Pacific Islander background.) 

O American Indian or Alaskan Native (“American Indian or Alaskan 
Native” means someone who is from one of the American Indian 
tribes or is one of the original people of Alaska.) 

O Other (specify) 



Students’ race/ethnicity was then assigned on the basis of their responses. For students 
who filled in the sixth oval (“Other”), provided illegible information or information that could 
not be classified, or who did not respond at all, race/ethnicity was assigned as determined by 
school records . 6 

Race/ethnicity could not be determined for students who did not respond to either of the 
demographic questions and whose schools did not provide information about race/ethnicity. 



6 The procedure for assigning race/ethnicity was modified for Hawaii. See the NAEP 1996 Technical Report for details. 

222 

Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



ash 



Details of how race/ethnicity classifications were derived are presented so that readers can 
determine how useful the results are for their particular purposes. Also, some students indicated 
that they were from a Hispanic background (e.g., Puerto Rican or Cuban) and that a racial/ethnic 
category other than Hispanic best describes them. These students were classified as Hispanic 
based on the rules described above. Furthermore, information from the schools did not always 
correspond to how students described themselves. Therefore, the racial/ethnic results presented 
in this report attempt to provide a clear picture based on several sources of information. 

tPtmirc&mils 9 JhmpIhcBSrt UcbwcbU <Bdlm£(sattm)m 

The variable representing the level of parental education is derived from responses to two 
questions from the set of general student background questions. Students were asked to indicate 
the extent of their mother’s education and their father’s education. 



How far in school did your mother go? 

0 She did not finish high school. 

OShe graduated from high school. 

OShe had some education after high school. 
OShe graduated from college. 

0 1 don’t know. 



How far in school did your father go? 

O He did not finish high school. 

0 He graduated from high school. 

OHe had some education after high school. 
OHe graduated from college. 

0 1 don’t know. 



The information was combined into one parental education reporting variable determined 
through the following process. If a student indicated the extent of education for only one parent, 
that level was included in the data. If a student indicated the extent of education for both parents, 
the higher of the two levels was included in the data. If a student did not know the level of 
education for both parents or did not know the level for one parent and did not respond to the 
other, the parental education level was classified as “I don’t know.” If the student did not respond 
for either parent, the student was recorded as having provided no response. 
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Results are reported by the type of school that the student attended — public or nonpublic. 
Nonpublic schools include Catholic and other private schools. Although Bureau of Indian Affairs 
(BIA) schools and Department of Defense Domestic Dependent Elementary and Secondary 
Schools (DDESS) are not included in either the public or nonpublic categories, they are included 
in the overall national results. 



IRcBgjnaDm 

Results are reported for four regions of the nation: Northeast, Southeast, Central, and West. Figure 
A.l shows how states are subdivided into these regions. 






l Regions ©ff fffe €©mi Ivy 



THE NATION’S 



REPORT 

CARD 




[?tej{iOD©0gG' 


geffiCfeess!} 


SsooCweD 


WseR 


Connecticut 

Delaware 

District of Columbia 

Maine 

Maryland 

Massachusetts 

New Hampshire 

New Jersey 

New York 

Pennsylvania 

Rhode Island 

Vermont 

Virginia* 


Alabama 

Arkansas 

Florida 

Georgia 

Kentucky 

Louisiana 

Mississippi 

North Carolina 

South Carolina 

Tennessee 

Virginia* 

West Virginia 


Illinois 

Indiana 

Iowa 

Kansas 

Michigan 

Minnesota 

Missouri 

Nebraska 

North Dakota 

Ohio 

South Dakota 
Wisconsin 


Alaska 

Arizona 

California 

Colorado 

Hawaii 

Idaho 

Montana 

Nevada 

New Mexico 

Oklahoma 

Oregon 

Texas 

Utah 

Washington 

Wyoming 



* Note: The part of Virginia that is included in the Washington, DC metropolitan area is included in the Northeast region; 
the remainder of the state is included in the Southeast region. 



Based on available school records, students were classified either as currently participating in a 
Title 1 program or receiving Title 1 services, or as not receiving such services. The classification 
applies only to the school year when the assessment was administered (i.e., the 1995-96 school 
year) and was not based on participation in previous years. If the school did not offer Title 1 
programs or services, all students in that school would be classified as not participating. 
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Based on available school records, students were classified as either currently eligible for the 
Free/Reduced-Price School Lunch component of the Department of Agriculture’s National School 
Lunch Program or not eligible. The classification applies only to the school year when the 
assessment was administered (i.e., the 1995-96 school year) and was not based on eligibility in 
previous years. If school records were not available, the student was classified as “Information not 
available.” If the school did not participate in the program, all students in that school were 
classified as “Information not available.” 



( UaamttnaDm® o m UmdcB^jp^cstlGatlnoDm 

This report describes students’, teachers’, and principals’ responses to background questions as 
well as mathematics performance for fourth-, eighth-, and twelfth-grade students. The report also 
compares the performance results for various groups of students within these populations (e.g., 
those who responded to a specific background question in a particular way or by individual 
course-taking groups as described above). However, it does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. In 
interpreting these data, it is important to understand that a relationship that exists between 
achievement and another variable does not reveal its underlying cause, which may be influenced 
by a number of other variables. Similarly, the assessments do not capture the influence of 
unmeasured variables. The results are most useful when they are considered in combination with 
other knowledge about the student population and the educational system, such as trends in 
instruction, changes in the school-age population, and societal demands and expectations. 

LEsdnmaadnmgj Waav°n(ialbMndQ[j 

Because the statistics presented in this report are estimates of group and subgroup performance 
based on samples of students rather than the estimates that could be calculated if every student in 
the nation answered every question, the degree of uncertainty associated with the estimates 
should be taken into account. Two components of uncertainty are accounted for in the variability 
statistics based on student ability: (1) the uncertainty due to sampling only a relatively small 
number of students and (2) the uncertainty due to sampling only a relatively small number of 
cognitive questions. The first component accounts for the variability associated with the estimated 
percentages of students who had certain background characteristics or who answered a certain 
cognitive question correctly. 

Because NAEP uses complex sampling procedures, conventional formulas for estimating 
sampling variability that assume simple random sampling are inappropriate. NAEP uses a 
jackknife replication procedure to estimate standard error. The jackknife standard error provides 
•a reasonable measure of uncertainty for any student information that can be observed without 
error. However, because each student typically responds to only a few questions within any 



221 



a 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



A =24 



content strand, the scale score for any single student would be imprecise. In this case, plausible 
values technology can be used to describe the performance of groups or subgroups of students, 
but the underlying imprecision involved in this step adds another component of variability to 
statistics based on NAEP scale scores. 7 

Typically, when the standard error is based on a small number of students or when the 
group of students is enrolled in a small number of schools, the amount of uncertainty associated 
with the standard error may be quite large. Throughout this report, estimates of standard errors 
subject to a large degree of uncertainty are designated. 

The reader is reminded that, like findings from all surveys, NAEP results are subject to 
other kinds of error, including the effects of imperfect adjustments for student and school 
nonresponse and unknowable effects associated with the particular instrumentation and data 
collection methods. Nonsampling errors can be attributed to a number of sources — inability to 
obtain complete information about all selected schools in the sample (some students or schools 
refused to participate, or students participated but answered only certain questions); ambiguous 
definitions; differences in interpreting questions; inability or unwillingness to give correct 
information; mistakes in recording, coding, or scoring data; and other errors in collecting, 
processing, sampling, and estimating missing data. The extent of nonsampling error is difficult to 
estimate, and, because of their nature, the impact of such errors cannot be reflected in the data- 
based estimates of uncertainty provided in NAEP reports. 



Wv°Gamnmg[j UrnffcBtrcBmcEcB® firraDm 

As noted, because the percentages of students and their average scale scores are based on 
samples rather than on the entire population of fourth-, eighth-, or twelfth-graders in the nation or 
a jurisdiction, the numbers reported are estimates. As such, they are subject to a measure of 
uncertainty, reflected in the standard error of the estimate. When the percentages or average scale 
scores of certain groups are compared, the standard error should be taken into account, and 
observed similarities or differences should not be relied on solely. Therefore, the comparisons 
discussed in this report are based on statistical tests that consider the standard errors of those 
statistics and the magnitude of the difference among the averages or percentages. 

The results from the sample, taking into account the uncertainty associated with all 
samples, are used to make inferences about the population. Using confidence intervals based on 
the standard errors provides a way to make inferences about the population averages and 
percentages in a manner that reflects the uncertainty associated with the sample estimates. 

An estimated sample average scale score ± 2 standard errors approximates a 95 percent 
confidence interval for the corresponding population quantity. This statement means that one can 
conclude with approximately a 5 percent level of confidence that the average performance of the 
entire population of interest (e.g., all fourth-grade students in public schools in a jurisdiction) is 
within ± 2 standard errors of the sample average. 



7 For more details, see Johnson, E. G. & Rust, K. F. (1992). Population inferences and variance estimation for NAEP data. 
Journal of Educational Statistics , 1 7(2), 175-190. 
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As an example, suppose that the average mathematics scale score of the students in a 
particular group was 256, with a standard error of 1.2. A, 95 percent confidence interval for the 
population quantity could be described in any of the following ways: 

Average ± 2 standard errors 
256 ±2 X 1.2 
256 ±2.4 
253.6, 258.4 

Thus, one can conclude with a 5 percent level of confidence that the average scale score for the 
entire population of students in that group is between 253.6 and 258.4. 

Similar confidence intervals can be constructed for percentages, if the percentages are not 
extremely large or extremely small. For extreme percentages, confidence intervals constructed in 
the above manner may not be appropriate, and accurate confidence intervals can be constructed 
only using procedures that are quite complicated. 

Extreme percentages, defined by both the magnitude of the percentage and the size of the 
sample from which it was derived, should be interpreted with caution. The NAEP 1996 Technical 
Report contains a more complete discussion of extreme percentages. 8 



MmmUgjgnmgj (BtrcDmjp WnHHcBtrcBmascBs nm AlwcBiraagpcBs 
aamdl 

The statistical tests determine whether the evidence, based on the data from the groups in the 
sample, is strong enough to conclude that the averages or percentages are actually different for 
those groups in the population. If the evidence is strong (i.e., the difference is statistically 
significant), the report describes the group averages or percentages as being different 
(e.g., one group performed higher than or lower than another group), regardless of whether the 
sample averages or percentages appear to be approximately the same. If the evidence is not 
sufficiently strong (i.e., the difference is not statistically significant), the averages or percentages 
are described as being not significantly different, regardless of whether the sample averages or 
percentages appear to be approximately the same or widely discrepant. 

The reader is cautioned to rely on the results of the statistical tests rather than on the 
apparent magnitude of the difference between sample averages or percentages when determining 
whether the sample differences are likely to represent actual differences among the groups in 
the population. 

To determine whether a real difference exists between the average scale scores 
(or percentages of a certain attribute) for two groups in the population, one needs to obtain an 
estimate of the degree of uncertainty associated with the difference between the averages (or 
percentages) of these groups for the sample. This estimate of the degree of uncertainty, called the 
standard error of the difference between the groups, is obtained by taking the square of each group’s 
standard error, summing the squared standard errors, and taking the square root of that sum. 



8 Allen, et al., (1999). op. cit. 
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Similar to how the standard error for an individual group average or percentage is used, 
the standard error of the difference can be used to help determine whether differences among 
groups in the population are real. The difference between the averages or percentages of the two 
groups ± 2 standard errors of the difference represents an approximate 95 percent confidence 
interval. If the resulting interval includes zero, there is insufficient evidence to claim that a real 
difference between the groups is statistically significant (different) at the 0.05 level. In this report, 
differences among groups that involve poorly defined variability estimates or extreme percentages 
are not discussed. 

As an example, to determine whether the average mathematics scale score of Group A is 
higher than that of Group B, suppose that the sample estimates of the average scale score and 
standard errors were as follows: 

Group Average Scale Score Standard Error 

A 218 0.9 

B 216 1.1 

The difference between the estimates of the average scale scores of Groups A and B is 2 points 
(218—216). The standard error of this difference is: 

V(0.9 2 + IF) = 1.4 

Thus, an approximate 95 percent confidence interval for this difference is: 

Difference ± 2 standard errors of the difference 
2 ±2 X 1.4 
2 ± 2.8 
-0.8, 4.8 

The value zero is within the confidence interval; therefore, there is insufficient evidence to claim 
that Group A outperformed Group B. 

The procedures described in this section and the certainty ascribed to intervals 
(e.g., a 95 percent confidence interval) are based on statistical theory that assumes that only one 
confidence interval or test of statistical significance is being performed. However, in Chapters 2^1 
of this report, many different groups are being compared (i.e., multiple sets of confidence intervals 
are being analyzed). In sets of confidence intervals, statistical theory indicates that the certainty 
associated with the entire set of intervals is less than that attributable to each individual 
comparison from the set. To hold the significance level for the set of comparisons at a particular 
level (e.g., 0.05), adjustments (called multiple comparison procedures) must be made to the 
methods described in the previous section. One such procedure, the Bonferroni method, was 
used in the analyses described in this report to obtain confidence intervals for the differences 
among groups when sets of comparisons were considered. 9 Thus, the confidence intervals for the 
sets of comparisons in the text are more conservative than those described on the previous pages. 

9 Miller, R. G. (1966). Simultaneous statistical inference. New York: 
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Most of the multiple comparisons in this report pertain to relatively small sets or families 
of comparisons. For example, for discussions concerning comparisons of parents’ level of 
education, six comparisons were conducted — all pairs of the four parental education levels. 

In these situations, Bonferroni procedures were appropriate. 
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The comparisons presented in this report are based on statistical tests that consider the magnitude 
of the difference between group averages or percentages and the standard errors of those statistics. 
The following appendix contains the standard errors for the averages and percentages discussed 
in Chapters 2, 3, and 4. For ease of reference, the format and headings of each table in this 
appendix match the corresponding chapter table, although the numbers that appear are actually 
standard errors. 



Table B2.4 Standard Errors for Average Scale Scores for National 

NAEP and Estimation Studies, Grades 4, 8, and 12 B-5 

Table B2.5 Standard Errors for Scale Scores in Estimation by 

Background Variables, Grades 4, 8, and 12, 1996 B-6 

Table B2.6 Standard Errors for Average Scale Scores in Estimation 

at Different Percentile Levels, Grades 4, 8, and 12 B-7 

Table B2.7 Standard Errors for National Percentages Attaining 

Achievement Levels in Estimation, Grades 4, 8, and 12 B-8 

Table B2.8 Standard Errors for Percent of Students Reaching 

at Least Proficient Level in Estimation by Background 

Variables, Grades 4, 8, and 12, 1996 B-9 

Table B3.1 Standard Errors for Student Demographic Distributions 

by Assessment, Grade 4, 19^6 B-10 

Table B3.2 Standard Errors for Percentage of Students by Teachers’ 

Reports on Classroom Practices, Grade 4, 1996 B-ll 

Table B3.3 ' Standard Errors for Average Percentage Correct Scores 

by Theme Block, Grade 4, 1996 B-12 

Table B3.4 Standard Errors for Score Percentages for 

“Draw Symmetrical Figure,” Grade 4 B-13 

Table B3.5 Standard Errors for Score Percentages for . 

“Measure Length Using Ruler,” Grade 4 B-13 

Table B3.6 Standard Errors for Score Percentages for 

“Solve Packing Problems,” Grade 4 B-14 

Table B3.7 Standard Errors for Score Percentages for 

“Determine Number of Models,” Grade 4 ; B-14 
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Table B3.8 Standard Errors for Score Percentages for 

“Determine Number of Leaves,” Grade 4 B-15 

Table B3.9 Standard Errors for Score Percentages for 

“Interpret Pattern of Figures,” Grade 4 B-15 

Table B3.10 Standard Errors for Student Demographic 

Distributions by Assessment, Grade 8, 1996 B-16 

Table B3.ll Standard Errors for Percentage of Students by 

Teachers’ Reports on Classroom Practices, Grade 8, 1996 B-17 

Table B3.12 Standard Errors for Average Percentage Correct Scores 

by Theme Block, Grade 8, 1996 B-18 

Table B3.13 Standard Errors for Score Percentages for 

“Identifying Needed Information,” Grade 8 B-19 

Table B3.14 Standard Errors for Percentages Correct for 

“Identifying Needed Information,” Grade 8 B-19 

Table B3.15 Standard Errors for Percentage Correct for 

“Determine Minimum Measuring Needed,” Grade 8 B-20 

Table B3.16 Standard Errors for Score Percentages for 

“Measure Lengths Using Ruler,” Grade 8 B-20 

Table B3.17 Standard Errors for Score Percentages for 

“Apply Concept of Ratio,” Grade 8 B-21 

Table B3.18 Standard Errors for Percentage Correct for 

“Understand Concept of Ratio (I),” Grade 8 B-21 

Table B3.19 Standard Errors for Percentage Correct for 

“Understand Concept of Ratio (II),” Grade 8 B-22 

Table B3.20 Standard Errors for Score Percentages for 

“Correctly Position Door,” Grade 8 B-22 

Table B3.21 Standard Errors for Score Percentages for 

“Visualize Cut-Outs on Grid,” Grade 8 B-23 

Table B3.22 Standard Errors for Score Percentages for 

“Apply Geometry in Model,” Grade 8 B-23 

Table B3.23 Standard Errors for Score Percentages for 

“Find Maximum Area When Perimeter is Fixed,” Grade 8 B-24 

Table B3.24 Standard Errors for Student Demographic Distributions 

by Assessment, Grade 12, 1996 B-25 

Table B3.25 Standard Errors for Percentage of Students by 

Reports on Classroom Practices, Grade 12, 1996 B-26 

Table B3.26 Standard Errors for Average Percentage Correct 

Scores by Theme Block, Grade 12, 1996 B-27 

Table B3.27 Standard Errors for Percentage Correct for 

“Find Amount of Down Payment,” Grade 12 B-28 

Table B3.28 Standard Errors for Percentage Correct for 

“Find Total Amount Paid for Car,” Grade 12 B-28 

Table B3.29 Standard Errors for Percentage Correct for “Find Difference 

Between Total Amount Paid and Price,” Grade 12 B-29 
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Table B3.30 Standard Errors for Score Percentages for 

“Find Amount To Be Financed,” Grade 12 B-29 

Table B3.31 Standard Errors for Score Percentages for 

“Use Formula to Find Total Cost,” Grade 12 B-30 

Table B3.32 Standard Errors for Score Percentages for 

“Find Amount Saved if Leased,” Grade 12 B-30 

Table B3.33 Standard Errors for Score Percentages for 

“Price Lease vs. Buy,” Grade 12 B-31 

Table B4.1 Standard Errors for Student Demographic 

Distributions, Grade 8, 1996 B-32 

Table B4.2 Standard Errors for Student/School 

Demographic Distributions, Grade 8, 1996 B-33 

Table B4.3 Standard Errors for Content Emphases in 

Mathematics, Grade 8, 1996 B-34 

Table B4.4 Standard Errors for Process Emphases 

in Mathematics, Grade 8, 1996 B-35 

Table B4.5 Standard Errors for Calculator Emphases 

in Mathematics, Grade 8, 1996 B-36 

Table B4.6 Standard Errors for Average Mathematics Scale 

Scores by Eligibility for Advanced Study, Grade 8 B-36 

Table B4.8 Standard Errors for Average Percentage 

Correct Scores, Advanced Study, Grade 8, 1996 B-37 

Table B4.9 Standard Errors for Score Percentages for 

“Car Wash Supplies,” Grade 8 B-37 

Table B4.10 Standard Errors for Score Percentages for 

“Begin to Earn Profit,” Grade 8 B-38 

Table B4.ll Standard Errors for Score Percentages for 

“Greatest Profit Expected,” Grade 8 B-39 

Table B4.12 Standard Errors for Score Percentages for 

“Hot Air Balloon,” Grade 8 B-39 

Table B4.13 Standard Errors for Student Demographic 

Distributions, Grade 12, 1996 B-40 

Table B4.14 Standard Errors for Student/School Demographic 

Distributions, Grade 12, 1996 B-41 

Table B4.15 Standard Errors for Content Emphases in Classes 

Taken by Advanced Study Students, Grade 12, 1996 B-42 

Table B4.16 Standard Errors for Process Emphases in Classes 

Taken by Advanced Study Students, Grade 12, 1996 B-43 

Table B4.17 Standard Errors for Calculator Access in Classes 

Taken by Advanced Study Students, Grade 12, 1996 B-44 

Table B4.18 Standard Errors for Calculator Usage and Instruction 

Classes Taken by Advanced Study Students, Grade 12, 1996 B-45 

Table B4.19 Standard Errors for Average Mathematics Scale 

Scores by Eligibility for Advanced Study, Grade 12, 1996 B-46 
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Table B4.21 Standard Errors for Average Percentage Correct Scores, 

Advanced Study, Grade 12, 1996 B-48 

Table B4.22 Standard Errors for Score Percentages for 

“Use Linear Function,” Grade 12 B-49 

Table B4.23 Standard Errors for Score Percentages for 

“Compare Volumes of Pyramids, ’’Grade 12 B-50 

Table B4.24 Percentage Correct for “Find Resultant Vector,” Grade 12 B-51 

Table B4.25 Standard Errors for Score Percentages for 

“Ferris Wheel,” Grade 12 B-52 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



TfefeO© 



Hirr©r§ fer Mveirsige §<g<sifl® §s@r®s @@ir 
Ms/ifctfM ill INMhMP mdl Estam©fB&!n BfttmdUies, 
@ir@dl@§ $, B, gam gS H2 



THE NATION’S 






Assessment Year 


Average Overall 
Scale Score in 

Mathematics NAEP 


Average Estimation 
Scale Score 


©Kails^ 










1996 


0.9 


2.1 




1992 


0.7 


1.5 




1990 


0.9 


1.5 


©Kails © 










1996 


1.1 


1.2 




1992 


0.9 


1.3 




1990 


1.3 


1.2 


©kex»)s US 










1996 


1.0 


1.2 




1992 


0.9 


1.2 




1990 


1.1 


1.2 



[ 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1996 Mathematics Assessments. 
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THE NATION’S 














Grade 4 


Grade 8 


Grade 12 


Gender 








Males 


2.3 


1.7 


1.6 


Females 


2.3 


1.6 


1.4 


Students who Indicated Their 
Race/Ethnicity as... 








White 


2.5 


1.6 


1.4 


Black 


4.2 


2.0 


3.0 


Hispanic 


2.8 


2.8 


3.1 


Asian/Pacific Islander 


9.2! 


3.0 


6.1 


American Indian 


★ ★ ★ 


* * * 


★ ★ ★ 


Students who Reported Their 
Parents' Highest Level of 
Education as... 








Did Not Finish High School 


5.1 


2.6 


4.2 


Graduated From High School 


3.6 


1.7 


1.7 


Some Education After High School 


5.6 


1.5 


1.7 


Graduated From College 


2.9 


1.9 


1.6 


1 Don't Know 


2.7 


2.0 


* * * 


Students who Attend... 








Public Schools 


2.2 


1.3 


1.2 


Nonpublic Schools 


4.9! 


. 3.6! 


4.4! 


Title 1 Participation... 








Participated 


3.2 


3.2 


4.9! 


Did Not Participate 


2.3 


1.4 


1.3 


Free/Reduced-Price Lunch 
Program Eligibility... 








Not Eligible 


2.1 


1.9 


1.4 


Eligible 


2.8 


1.7 


3.1 


Information Unavailable 


5.8 


2.8! 


2.8 


i “ ' ‘ 









*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Sitamdmdl ISsrmirs ff@ir @sver<®ge 5<s<sile Semes m 
®f Doffiferenlt PereemVale Leveh/ 
©mdes 4/ 8, &nd 12 



= THE MATIOM’S 
REPORT 
CARO 



Iff 





Assessment 

Year 


10th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


90th 

Percentile 


















1996 


5.5 


2.6 


1.5 


3.0 


3.6 




1992 


1.5 


2.4 


1.8 


1.5 


3.1 




1990 


2.4 


2.0 


2.6 


1.9 


2.4 


















1996 


1.2 


1.4 


1.7 


1.1 


2.5 




1992 


2.1 


2.0 


1.6 


1.9 


2.5 




1990 


1.3 


1.8 


2.3 


2.3 


1.2 


©ornate 
















1996 


1.2 


1.9 


1.6 


1.2 


1.7 




1992 


1.4 


1.9 


1.5 


0.9 


2.1 




1990 


1.5 


1.6 


2.2 


1.3 


1.1 



[ 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1996 Mathematics Assessments. 
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m-r 




{?©ir B^®QS@ini©l P®ir<s®!mf®ges Mframmg) report ' t ^^ 

rmmmO fa gBS'ijm@ftS©ini / CAR n ^ 



<§r®dle§ 4, 8, ©md H2 







Assessment 

Year 


Advanced 


At or Above 
Proficient 


At or Above 
Basic 


Below Sasic 
















1996 


0.4 


2.4 


1.3 


1.3 




1992 


0.3 


1.7 


1.1 


1.1 




1990 


... 


1.3 


1.8 


1.8 


@K3(o]s © 














1996 


0.3 


1.6 


1.9 


1.9 




1992 


0.5 


1.8 


2.0 


2.0 




1990 


0.5 


1.6 


2.3 


2.3 


©JGXofe t)g 














1996 


1.0 


2.5 


1.2 


1.2 




1992 


0.7 


2.0 


1.7 


1.7 




1990 


0.9 


2.1 


1.6 


1.6 



[ 



--- Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 1992, and 
1 996 Mathematics Assessments. 
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Staffidfems? Srmrs Percent Shttforote Reaching 
of teosf Proficient Level m Estimation by 
Background Variables, Grades 4, 8, @md 12, 1996 



= THE NATION'S 
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CARO 



TOH[5] 





RsaaDGcp ®G SGbI&dGs RRsOBfeoD ca? (DsOOar 




Grade 4 


Grade 8 


Grade 12 


Gender 








Males 


3.1 


2.0 


2.7 


Females 


2.7 


2.1 


3.2 


Students who Indicated Their 
Race/Ethnicity as... 








White 


3.1 


2.1 


3.1 


Black 


2.6 


1.1 


3.9 


Hispanic 


2.4 


2.4 


3.6 


Asian/Pacific Islander 


9.9! 


3.5 


8.2 


American Indian 


9.9! 




... 


Students who Reported Their 
Parents' Highest Level of 
Education as... 








Did Not Finish High School 


4.2 


3.1 


4.6 


Graduated From High School 


4.0 


2.3 


3.1 


Some Education After High School 


5.9 


2.9 


3.5 


Graduated From College 


4.0 


2.7 


3.0 


1 Don't Know 


2.9 


2.1 


... 


Students who Attend... 








Public Schools 


2.6 


1.7 


2.2 


Nonpublic Schools 


5.8! 


5.2! 


7.4! 


Title 1 Participation... 








Participated 


2.3 


— 


4.2! 


Did Not Participate 


2.6 


1.7 


2.6 


Free/Reduced-Price Lunch 
Program Eligibility... 








Not Eligible 


3.1 


2.8 


2.7 


Eligible 


2.4 


1.9 


3.4 


Information Unavailable 


6.1 


2.7! 


4.4 


i 









-** Standard errors could not be accurately determined. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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TfefefeBSot] 



Si ?<§midl<§iirdl lErtmtrs @©tr Stosfecui? S>em©gir©pbik 
l§>ii§{hriib(y)ftS©ini§ by ^ssessmemft, (@ir©<dle 4, H ^3 



THE NATION’S 






0 O SttnilaiOs 






Theme Slock 1 


Theme Bloch 2 




Main Assessment 


The Sufferflv Booth 


Recycling 


@G£X°te 4J 








Gender 








Males 


0.7 


1.4 


1.4 


Females 


0.7 


1.4 


1.4 


Students who Indicated Their 








Race/Ethnicity as... 








White 


0.4 


0.9 


0.9 


Black 


0.2 


0.6 


0.6 


Hispanic 


0.4 


0.6 


0.7 


Asian/Pacific Islander 


0.2 


0.3 


0.4 


American Indian 


0.2 


0.3 


0.3 


Students who Reported Their 








Parents' Highest Level of . 








Education as... 








Did Not Finish High School 


0.3 


0.5 


0.8 


Graduated From High School 


0.6 


1.0 


1.0 


Some Education After High School 


0.4 


0.7 


0.7 


Graduated From College 


1.2 


1.3 


1.4 


1 Don't Know 


0.9 


1.1 


1.2 


Students who Attend... 








Public Schools 


1.6 


1.8 


1.6 


Nonpublic Schools 
Title 1 Participation... 


1.6 


1.8 


1.6 


Participated 


1.4 


1.9 


1.9 


Did Not Participate 


■ 1.4 


1.9 


1.9 


Free/Reduced-Price Lunch 








Program Eligibility... 








Eligible 


1.4 


1.6 


1.3 


Not Eligible 


2.5 


2.7 


2.6 


Information Not Available 


3.0 


2.4 


2.3 


i 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 




239 



m-i ® 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



Tfofelte E@o2 



Sft@)in)d!@}[rdl Eirmn fe/r IPemseinifrgige ©ff Sto<§fe/fi)i?s fey 
T<s<Q}£lh(S[r§' E£®p©ir9§ ©mi €B©5§[r©©m Pvadlkes, 
©{r©de $, 



THE WATIDN’S 






IJfeDC33CDOlS0© 0C §0ax£blD8 


Main Assessment 


Theme Block 1 
The Butterfly Booth 


Theme Block 2 
Recycling 


®oodte G> 








Students Whose Teachers Report 








Asking Students to Write a Few 








Sentences About How to Solve a 








Mathematics Problem... 








Nearly Every Day 


1.4 


1.2 


0.9 


Once or Twice a Week 


2.1 


2.1 


2.4 


Once or Twice a Month 


2.6 


2.3 


2.7 


Never or Hardly Ever 


2.4 


2.8 


3.0 


Students Whose Teachers Report 








Asking Students to Write Reports 








or Do a Mathematics Project... 








Nearly Every Day 


0.5 


0.1 


0.1 


Once or Twice a Week 


0.8 


0.6 


0.5 


Once or Twice a Month 


2.3 


2.7 


2.6 


Never or Hardly Ever 


2.4 


2.7 


2.7 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Staffl<§7@/r<§7 lEirmirs fe/r Avenge /Pe/rce/ra tag/e C®/r/r©ef 
Scores by Theme B§mk, ©mde 4J, U <§><§><§> 



= THE NATION’S 
REPORT 
CARD 






The Butterfly Booth 


Recycling 








All Students 


0.7 


0.6 


Gender 






Males 


0.9 


0.8 


Females 


0.8 


0.6 


Race/Ethnicity 






White 


0.8 


0.7 


Black 


0.7 


1.1 


Hispanic 


0.9 


1.1 


Asian/Pacific Islander 


2.4 


2.1 


American Indian 


★ ★ ★ 


* * * 


Students Whose Teachers Report Asking 






Students to Write a Few Sentences 






About How to Solve a 






Mathematics Problem... 






Nearly Every Day 


3.3 


1.9 


Once or Twice a Week 


1.2 


1.3 


Once or Twice a Month 


1.2 


1.4 


Never or Hardly Ever 


1.2 


1.3 


Students Whose Teachers Report Asking 






Students to Write Reports or Do a 






Mathematics Project... 






Nearly Every Day 


★ ★ ★ 


★ * * 


Once or Twice a Week 


★ * * 


★ * * 


Once or Twice a Month 


1.4 


1.6 


Never or Hardly Ever 


0.8 


0.7 


L 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Stantons? Eirir@irs §@ir Bs@re Petreemkages ff@r 
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THE NATION’S 



REPORT 

CARD 



ft 





Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


@(KD^fe<3 












All Students 


1.2 


0.8 


0.9 


0.7 


1.4 


Males 


1.6 


1.1 


1.3 


1.1 


1.8 


Females 


• 1.8 


1.3 


1.2 


0.9 


1.7 


White 


1.4 


0.9 


1.1 


0.9 


1.5 


Black 


1.7 


1.8 


2.1 


1.9 


2.6 


Hispanic 


1.7 


2.0 


1.5 


1.5 


2.9 


Asian/Pacific Islander 


7.6 


4.1 


3.1 


2.8 


5.5 


American Indian 


* * * 


* * * 


* * * 


* * * 


* * * 



c 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Sfo/nxsJord Errors fer §<s@re Pereem^ges ff@r 
"M essure lemgitih HJJsSmgj P&sller," ©mde 4J 
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s 





Extended 


Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


@0*3^3 














All Students 


1.4 


0.8 


1.0 


0.5 


1.3 


0.5 


Males 


2.2 


1.2 


1.4 


0.5 


2.0 


0.8 


Females 


1.7 


1.4 


1.1 


0.9 


1.4 


0.6 


White 


1.7 


1.0 


1.2 


0.7 


1.6 


0.4 


Black 


2.0 


2.3 


2.0 


1.3 


2.7 


1.5 


Hispanic 


2.7 


1.5 


2.2 


0.9 


3.2 


2.3 


Asian/Pacific Islander 


6.0 


2.8 


2.5 


1.3 


5.3 


1.6 


American Indian 


★ * * 


* * * 


* * * 


* * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 




Sft(§iinidl(§W‘<dl lEu‘\r@u‘§ @@tr §€@ir<s (P<&ir<g®inift@g[j@§ @@tr 


REPORT 

CARO 




"§@Sv<b IPmbBems," ©r@dl® $ 






i: 





Extended 


Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 
















All Students 


0.7 


1.0 


1.1 


1.4 


1.3 


0.2 


Males 


1.2 


1.3 


1.2 


1.8 


1.7 


0.4 


Females 


0.9 


1.3 


1.6 


1.7 


1.8 


0.2 


White 


0.9 


1.3 


1.5 


1.9 


1.8 


0.3 


Black 


1.0 


0.9 


1.4 


2.6 


2.4 


0.6 


Hispanic 


0.4 


1.6 


1.8 


2.9 


3.2 





Asian/Pacific Islander 


1.4 


6.4 


4.3 


5.8 


4.4 




American Indian 

( 


* * * 


★ * * 


★ ★ ★ 


* * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

- - - Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 
REPORT Lv™ I 



CARO 



H: 





Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 














All Students 


0.4 


1.0 


1.0 


1.7 


0.4 


Males 


0.5 


1.3 


1.3 


2.0 


0.7 


Females 


0.5 


1.5 


1.4 


2.2 


0.4 


White 


0.6 


1.3 


1.4 


2.2 


0.5 


Black 


0.6 


1.2 


1.7 


2.5 


1.0 


Hispanic 


0.4 


1.5 


2.0 


2.3 


1.1 


Asian/Pacific Islander 


2.6 


4.4 


5.0 


6.0 


1.9 


American Indian 


* * * 


* * * 


* * * 


* * * 


* * * 


i 



Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 




Snrairs 1 fer §€@ire Perseimttages RE card 




TMteBSb© 


"LPeffermme Number Leaves/' ©radle 4 




§: 





Complete 


Partial 


Incorrect 


Omit 


©Knife <3 










All Students 


0.6 


0.8 


0.9 


0.3 


Males 


0.9 


0.9 


1.1 


0.4 


Females 


0.7 


1.1 


1.4 


0.4 


White 


0.7 


1.1 


1.3 


0.3 


Black 


1.0 


0.9 


1.8 


1.1 


Hispanic 


0.8 


0.8 


1.4 


0.8 


Asian/Pacific Islander 


3.2 


3.3 


4.9 


3.0 


American Indian 

1 


* * * 


★ * * 


* * * 


* * * 



*** Sample size is insufficient ta permit a reliable estimate. 

SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Standard Srrers for Bs@re Peireetnftages fof* "card 




‘fcfelfe ©So® 


"Inti’erprefr Pa Kern ©§ IFngures, " ©ra die $ 




3 





Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


©Knife 3 












All Students 


0.3 


0.4 


0.5 


0.8 


0.2 


Males 


0.4 


0.6 


0.7 


1.2 


0.3 


Females 


0.3 


0.6 


0.6 


1.0 


0.4 


White 


0.4 


0.6 


0.7 


1.1 


0.3 


Black 


— 


0.7 


1.2 


1.6 


0.6 


Hispanic 




0.5 


1.1 


2.1 


0.6 


Asian/Pacific Islander 






3.2 


3.7 


2.2 


American Indian 

1 


* * * 


* * * 


* * * 


* * * 


* * * 



*** Sample size is insufficient ta permit a reliable estimate. 

- - - Standard errors cauld nat be accurately determined. 

SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Main Assessment 


Theme Slock 1 
Building a Doghouse 


Theme Slock 2 
Flooding 


©cods © 








Gender 








Males 


0.8 


1.0 


1.2 


Females 


0.8 


1.0 


1.2 


Race/Ethnicity 








White 


0.2 


0.7 


0.6 


Black 


0.2 


0.6 


0.5 


Hispanic 


0.1 


0.5 


0.5 


Asian/Pacific Islander 


0.2 


0.3 


0.3 


American Indian 


0.2 


0.3 


0.3 


Students who Reported Their 








Parents' Highest Level of 








Education as... 








Did Not Finish High School 


0.4 


0.7 


1.1 


Graduated From High School 


0.8 


1.2 


1.1 


Some Education After High School 


0.7 


1.0 


1.0 


Graduated From College 


1.3 


1.6 


1.7 


1 Don't Know 


0.6 


0.7 


0.7 


Students who Attend... 








Public Schools 


1.1 


1.9 


1.9 


Nonpublic Schools 


1.1 


1.9 


1.9 


Title 1 Participation... 








Participated 


1.6 


1.7 


1.7 


Did Not Participate 


1.6 


1.7 


1.7 


Free/Reduced-Price Lunch 








Program Eligibility... 








Eligible 


1.4 


1.5 


2.0 


Not Eligible 


2.4 


2.5 


2.8 


Information Not Available 

i 


2.9 


2.9 


3.0 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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©mde 8, 



THE NATION’S 



REPORT 

CARD 






fffeDQSDdOag]© ©OcffDaiDs 






Theme Block 1 


Theme Block 2 




Main Assessment 


Building o Doghouse 


Flooding 










Students Whose Teachers Report 








Asking Students to Write a Few 
Sentences About How to Solve 








a Mathematics Problem... 








Nearly Every Day 


i.i 


1.1 


1.2 


Once or Twice a Week 


2.8 


3.2 


3.7 


Once or Twice a Month 


2.8 


3.2 


3.6 


Never or Hardly Ever 


3.3 


3.4 


3.6 


Students Whose Teachers Report 
Asking Students to Write Reports 








or Do a Mathematics Project... 








Nearly Every Day 


... 


— 


— 


Once or Twice a Week 


1.1 


1.3 


1.4 


Once or Twice a Month 


3.2 


2.7 


2.8 


Never or Hardly Ever 
i = 


3.3 


2.5 


2.8 



— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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= 17 



TfeM§d@o^g 



${t©indl©irdl Eirmirs fair 
§<g©ir®§ by Yh®m 



Avetrsigje P®rs®in>^©g® €©mr®<s 9 
© IBfask, ©ir©d® B, H^>^> 



THE NATION’S 






Suild g Doghouse 


Flooding 


© 






All Students 


0.5 


0.7 


Gender 






Males 


0.6 


0.8 


Females 


0.7 


0.9 


Race/Ethnicity 






White 


0.7 


1.0 


Black 


0.8 


0.5 


Hispanic 


0.8 


0.8 


Asian/Pacific Islander 


2.5 


2.8 


American Indian 


* ★ ★ 


* * * 


Students Whose Teachers Report Asking 
Students to Write a Few Sentences 
About How to Solve 
a Mathematics Problem... 






Nearly Every Day 


3.1 


3.0 


Once or Twice a Week 


1.6 


2.4 


Once or Twice a Month 


1.1 


1.3 


Never or Hardly Ever 


1.0 


1.4 


Students Whose Teachers Report Asking 






Students to Write Reports or Do a 






Mathematics Project... 






Nearly Every Day 


* * * 


★ * * 


Once or Twice a Week 


2.6 


2.1 


Once or Twice a Month 


1.1 


1.4 


Never or Hardly Ever 

1 


0.9 


1.0 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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5 Correct 


4 Correct 


3 Correct 


2 Correct 


1 Correct 


0 Correct 


Omit 


©RSXife © 
















All Students 


1.5 


1.0 


0.7 


0.7 


0.7 


0.3 


0.1 


Males 


1.8 


1.3 


1.2 


0.9 


1.2 


0.5 


0.1 


Females 


2.1 


1.5 


0.9 


0.9 


0.8 


0.2 


0.1 


White 


1.9 


1.1 


0.8 


0.7 


0.7 


0.4 


— 


Black 


2.5 


2.0 


2.4 


1.4 


1.2 


0.5 


0.5 


Hispanic 


2.5 


2.8 


2.0 


1.7 


1.5 




0.3 


Asian/Pacific Islander 


5.2 


4.4 




2.5 


... 


... 


... 


American Indian 


* * * 


* * * 


* * * 


* * * 


* * * 


* * * 


* * * 

— 



*** Sample size is insufficient to permit a reliable estimate. 

— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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la - Yes 


1b - Yes 


1c - Wo 


Id — Yes 


1e - Yes 


@CK3^fe © 












All Students 


0.9 


0.9 


1.2 


1.0 


1.2 


Males 


1.6 


1.3 


1.5 


1.6 


1.6 


Females 


0.9 


1.1 


1.6 


0.9 


1.3 


White 


1.1 


1.1 


1.7 


1.2 


1.5 


Black 


2.2 


1.9 


1.9 


2.4 


2.5 


Hispanic 


1.8 


1.6 


3.0 


2.6 


2.7 


Asian/Pacific Islander 


— 


3.1 


4.1 


4.5 


4.1 


American Indian 


* * * 


* * * 


★ * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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"Beteraiffne Mffiiraisi!! We@<§fe<§7 / // ©ir^dlo 
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CARD 






Percentage Correct 


@R3C°te © 




All Students 


1.3 


Males 


1.3 


Females 


2.2 


White 


1.6 


Black 


2.6 


Hispanic 


2.4 


Asian/Pacific Islander 


7.4 


American Indian 

r 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
OMathematics Assessment. 
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"Meioistuiir® lemgttlhs UJsmg Ifttwlleir/' ©irtside 8 



THE NATION’S 






Complete 


Partial 


Incorrect 


Omit 


@K°X°te © 










All Students 


1.5 


1.2 


1.2 


1.3 


Males 


1.8 


1.4 


1.5 


1.2 


Females 


2.0 


1.7 


1.6 


1.6 


White 


1.8 


1.5 


1.4 


1.4 


Black 


2.4 


3.1 


4.1 


2.7 


Hispanic 


3.0 


3.0 


2.9 


1.5 


Asian/Pacific Islander 


7.3 


5.8 


8.5 





American Indian 


* * * 


* * * 


* * * 


* * * 


L 



*** Sample size is insufficient to permit a reliable estimate. 

- - - Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



0 



ERIC 



ZB~g© 



24 9 



Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 






Sf@ffDs7@/rs7 IE 'mr@r§ i for 



k©re PeirsemMigjes @©tr 
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Complete 


Partial 


Incorrect 


Omit 


©CKSdfe © 










All Students 


i.i 


1.2 


1.5 


1.1 


Males 


1.4 


1.5 


1.7 


1.1 


Females 


1.2 


1.5 


2.0 


1.7 


White 


1.4 


1.5 


1.8 


1.4 


Black 


0.9 


1.1 


2.8 


3.3 


Hispanic 


1.9 


1.7 


3.1 


2.5 


Asian/Pacific Islander 


4.8 


6.6 


7.2 


4.5 


American Indian 


★ ★ ★ 


★ ★ ★ 


* ★ ★ 


★ ★ ★ 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 





THE NATION’S 


TfeM§ BSoD© 


§9©tnidl@[rdl lEs-mirs ff©ir Peir<seini9@ge €©irinss9 §©v "card 




"iJJin)<dl@irsft©Hn)dl €©imsep^ ©@ /Sofa© (ti)," ©msodl® 8 




1 





Percentage Correct 


© 




All Students 


1.3 


Males 


1.9 


Females 


1.3 


White 


1.6 


Black 


2.3 


Hispanic 


3.1 


Asian/Pacific Islander 


9.0 


American Indian 


★ ★ ★ 


1 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Percentage Correct 






All Students 


1.2 


Males 


1.7 


Females 


1.7 


White 


1.4 


Black 


2.4 


Hispanic 


2.8 


Asian/Pacific Islander 


8.2 


American Indian 


* ★ ★ 


i 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



TfeMs 



$^©!mdl@rdl Err@rs {for §<s@r® {P®r<5®nv9@g®§ for 
"€©rr®€ftty P©$M©tm \S>@@r," ©i md® 8 



THE NATION’S 






Complete 


Partial 


Incorrect 


Omit 


dk»x»te © 










All Students 


1.3 


1.2 


1.3 


0.8 


Males 


2.0 


1.8 


2.0 


1.3 


Females 


1.7 


1.4 


1.6 


0.9 


White 


1.8 


1.4 


1.7 


0.9 


Black 


1.1 


2.9 


2.7 


2.7 


Hispanic 


1.9 


3.3 


3.2 


2.4 


Asian/Pacific Islander 


4.5 


8.3 


6.6 


6.8 


American Indian 


* * ★ 


* * * 


★ * * 


★ ★ ★ 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Complete 


Partial 


Incorrect 


Omit 












All Students 


1.5 


0.6 


1.4 


0.8 


Males 


1.7 


0.9 


1.5 


1.1 


Females 


2.1 


1.2 


2.1 


1.1 


White 


1.9 


0.8 


1.7 


0.8 


Black 


2.2 


0.8 


2.3 


2.6 


Hispanic 


2.6 


1.3 


2.7 


3.0 


Asian/Pacific Islander 


8.6 


5.0 


6.4 


4.8 


American Indian 


* ★ ★ 


* * ★ 


* * * 


* * * 



c 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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nfcrUn 1 

Statrasfore? frr®rs for §s@r® [P®tr<semfi’ag®$ for card 


Retire] 


"AppSy ©e©m®Vry m M@del," Grade 8 




§ 





Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


© 












All Students 


0.5 


0.7 


0.8 


i.i 


1.6 


Males 


0.4 


1.0 


1.3 


2.1 


2.3 


Females 


0.8 


0.9 


1.2 


1.6 


1.5 


White 


0.7 


1.0 


1.1 


1.2 


2.0 


Black 


— 


— 


1.4 


3.4 


3.1 


Hispanic 


— 


1.1 


2.0 


3.8 


3.5 


Asian/Pacific Islander 


* * * 


* * * 


* * * 


* * * 


* * * 


American Indian 


* * * 


* * * 


★ * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

- * - Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Extended 


Satisfactory 


Partial 


(Minimal 


Incorrect 


Omit 
















All Students 


0.1 


0.0 


1.2 


0.6 


1.7 


1.3 


Males 


0.1 


— 


1.8 


1.0 


1.7 


1.7 


Females 


0.2 


0.1 


1.5 


0.4 


2.2 


2.0 


White 


0.1 




1.5 


0.8 


1.9 


1.4 


Black 


— 


— 


1.7 


0.9 


3.3 


3.1 


Hispanic 


— 


— 


2.6 


1.0 


3.2 


4.0 


Asian/Pacific Islander 


* * * 


* * * 


* * * 


* * * 


* * * 


* * * 


American Indian 


* * * 


* * * 


* * * 


* * * 


* * * 


* * * 



[ 



*** Sample size is insufficient to permit a reliable estimate. 

— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 








Moin Assessment 


Theme Slock 1 
Buying a Car 


Theme Block 2 
Flooding 


@K°x°te DIS 








Gender 








Males 


0.9 


1.4 


1.3 


Females 


0.9 


1.4 


1.3 


Race/Ethnicity 








White 


0.5 


0.8 


0.8 


Black 


0.4 


0.6 


0.6 


Hispanic 


0.4 


0.6 


0.6 


Asian/Pacific Islander 


0.4 


0.3 


0.4 


American Indian 


0.6 


0.2 


0.2 


Students who Reported Their 








Parents' Highest Level of 








Education as... 








Did Not Finish High School 


0.5 


0.8 


0.6 


Graduated From High School 


0.8 


1.6 


1.3 


Some Education After High School 


0.8 


1.1 


1.3' 


Graduated From College 


1.5 


1.8 


1.8 


1 Don't Know 


0.2 


0.5 


0.5 


Students who Attend... 








Public Schools 


1.6 


1.9 


2.0 


Nonpublic Schools 


1.5 


1.9 


2.0 


Title 1 Participation... 








Participated 


0.6 


0.8 


0.8 


Did Not Participate 


0.6 


0.8 


0.8 


Free/Reduced-Price Lunch 








Program Eligibility... 








Eligible 


1.3 


1.2 


1.1 


Not Eligible 


3.7 


3.5 


3.7 


Information Not Available 


3.8 


3.5 


3.6 



j 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Klein Assessment 


Buying e Car 


Flooding 










Students who Report Writing 








a Few Sentences About How to 








Solve a Mathematics Problem... 








Nearly Every Day 


0.4 


0.7 


0.7 


Once or Twice a Week 


0.5 


0.8 


0.9 


Once or Twice a Month 


0.6 


1.4 


0.9 


Never or Hardly Ever 


0.8 


1.6 


1.6 


Students who Report Writing 








Reports or Doing a 








Mathematics Project... 








Nearly Every Day 


0.2 


... 




Once or Twice a Week 


0.3 


0.5 


0.5 


Once or Twice a Month 


1.0 


1.6 


1.1 


Never or Hardly Ever 


1.0 


1.8 


1.3 



* Teachers of twelfth-grade students were not surveyed in the Main Assessment. 

- - - Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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m 





[Buying a Cor 


Flooding 








All Students 


1.1 


0.8 


Gender 






Males 


1.4 


1.1 


Females 


1.2 


1.1 


Race/Ethnicity 






White 


1.3 


1.0 


Black 


1.3 


1.1 


Hispanic 


2.1 


1.9 


Asian/Pacific Islander 


3.8 


3.3 


American Indian 


* ★ ★ 


★ ★ ★ 


Students Whose Teachers Ask Them 
to Write a Few Sentences 
About How to Solve 
a Mathematics Problem... 






Nearly Every Day 


4.4 


1.9 


Once or Twice a Week 


2.3 


1.8 


Once or Twice a Month 


2.1 


1.5 


Never or Hardly Ever 


1.1 


1.1 


Students Whose Teachers Ask Them 
to Write Reports or 
Do a Mathematics Project... 






Nearly Every Day 


★ ★ ★ 


★ ★ ★ 


Once or Twice a Week 


2.9 


2.7 


Once or Twice a Month 


2.0 


1.4 


Never or Hardly Ever 


1.4 


1.0 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Percentage Correct 


@K°X°fe 




All Students 


i.i 


Males 


1.6 


Females 


1.6 


White 


1.0 


Black 


3.1 


Hispanic 


3.3 


Asian/Pacific Islander 


4.1 


American Indian 


* * * 


i 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 






Percentage Correct 


@K°x°te H2 




All Students 


i.i 


Males 


1.5 


Females 


1.2 


White 


1.2 


Black 


3.4 


Hispanic 


3.8 


Asian/Pacific Islander 


4.5 


American Indian 


* * * 


L 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE; National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 






Percentage Correct 


©KDdte tig 




All Students 


1.2 


Males 


1.9 


Females 


1.4 


White 


1.3 


Black 


3.6 


Hispanic 


3.1 


Asian/Pacific Islander 


4.1 


American Indian 


★ * * 


I 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Complete 


Partial 


Incorrect 


Omit 


@Cr5K°fe tig 










All Students 


1.4 


1.5 


1.4 


0.6 


Males 


2.0 


1.8 


1.7 


1.2 


Females 


1.6 


1.8 


1.6 


0.6 


White 


1.7 


1.6 


1.8 


0.6 


Black 


2.5 


3.2 


2.9 


2.2 


Hispanic 


3.8 


3.5 


3.3 


2.0 


Asian/Pacific Islander 


5.9 


5.1 


3.5 


3.2 


American Indian 


* * * 


★ * * 


* * * 


* * * 



1 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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intended 


Satisfactory 


Partial 


{minimal 


Incorrect 


Omit 


(&K°x°b US 














All Students 


0.9 


1.1 


0.9 


0.8 


1.5 


0.8 


Males 


1.0 


1.1 


1.0 


1.2 


1.9 


1.2 


Females 


1.2 


1.6 


1.2 


1.1 


2.0 


1.1 


White 


1.2 


1.4 


1.2 


1.0 


1.9 


1.0 


Black 


0.8 


0.9 


1.3 


1.9 


2.9 


2.0 


Hispanic 


1.6 


2.7 


1.2 


2.8 


4.0 


1.9 


Asian/Pacific Islander 


3.9 


4.1 


2.5 


2.3 


5.8 


2.9 


American Indian 


* * ★ 


★ ★ * 


★ ★ ★ 


★ ★ ★ 


* * * 


* * * 


i 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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NT 





Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


@«3C°fe tig 












All Students 


1.6 


1.0 


1.3 


1.3 


0.5 


Males 


2.0 


1.6 


1.5 


1.9 


0,9 


Females 


1.9 


1.6 


1.9 


1.7 


0.7 


White 


1.8 


1.2 


1.5 


1.3 


0.6 


Black 


1.7 


2.6 


4.2 


3.3 


2.6 


Hispanic 


3.7 


3.2 


3.6 


3.9 


2.2 


Asian/Pacific Islander 


4.3 


3.9 


4.9 


7.1 


0.8 


American Indian 


* * * 


★ ★ ★ 


★ ★ ★ 


* * ★ 


•k k k 


V 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Satisfactory 


Partial 


Minimal 


Incorrect 


Omit 


US 












All Students 


1.5 


0.4 


0.9 


1.7 


0.9 


Males 


1.9 


0.7 


1.4 


2.7 


1.1 


Females 


1.7 


0.4 


1.4 


1.9 


1.2 


White 


1.8 


0.6 


1.1 


1.9 


0.9 


Black 


1.4 


— 


1.8 


4.1 


4.0 


Hispanic 


2.8 


... 


1.3 


3.9 


2.7 


Asian/Pacific Islander 


4.2 


1.5 


4.8 


7.0 


1.1 


American Indian 


* * * 


* * * 


★ * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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RarascaOsgs of} §0oi£lt3iC8 


Advanced Study 


Not Eligible lor 
Advanced Study 


@K3°fe © 






Gender 






Males 


1.2 


0.9 


Females 


1.2 


0.9 


Students who Indicated Their 






Race/Ethnicity as... 






While 


2.6 


0.7 


Black 


2.7 


0.5 


Hispanic 


0.9 


0.4 


Asian/Pacific Islander 


0.8 


0.5 


American Indian 


0.8 . 


0.5 


Students who Reported Their 






Parents' Highest Level 






of Education as... 






Did Not Finish High School 


0.4 


0.5 


Graduated From High School 


1.2 


0.8 


Some Education After High School 


1.0 


0.7 


Graduated From College 


1.6 


1.2 


1 Don't Know 


0.5 


0.8 


Home Environment Contains... 






0-2 types of educational materials 


1.1 


0.7 


3 types of educational materials 


1.2 


1.0 


4 types of educational materials 


1.4 


1.0 


Students From... 






Northeast 


3.8 


1.4 


Southeast 


2.2 


1.8 


Central 


4.0 


1.7 


West 


2.7 


1.7 


Students Live in... 






Center City 


4.2 


2.8 


Urban Fringe/Small City 


4.6 


3.1 


Rural 

i 


3.8 


3.2 

— 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Advanced Study 


Not Eligible {or 
Advanced Study 








Students who Attend... 






Public Schools 


2.3 


1.0 


Nonpublic Schools 


2.3 


1.0 


Title 1 Participation... 






Participated 


0.8 


1.9 


Did Not Participate 


0.8 


1.9 


Free/Reduced-Price Lunch 
Program Eligibility... 






Eligible 


2.0 


1.6 


Not Eligible 


3.3 


2.3 


Information Not Available 

1 


3.4 


2.8 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 


§9@[R}(Ql@[r<Ql IE inr@ir§ m card I 

M<siftlh®m<ai9k§, ©md® 8, 







GteGftip efi SftidaiCs 


None 


A Little 


Some 


A Lot 
















In This Mathematics Class 












How Often Do 












You Address... 












Numbers and Operations 












Advanced Study 


— 


1.8 


3.7 


3.7 




Non-Eligibles 




0.5 


1.8 


1.9 




Measurement 












Advanced Study 


1.6 


4.0 


4.5 


2.5 




Non-Eligibles 


0.3 


2.7 


3.7 


3.1 




Geometry 












Advanced Study 


3.1 


3.3 


3.6 


2.8 




Non-Eligibles 


0.7 


3.2 


3.4 


3.0 




Data Analysis; 












Statistics; and Probability 












Advanced Study 


3.1 


4.9 


5.0 


2.7 




Non-Eligibles 


1.4 


3.4 


3.5 


2.5 




Algebra and Functions 












Advanced Study 


— 


0.5 


2.2 


2.2 




Non-Eligibles 

i 


0.4 


1.4 


3.5 


3.8 





— Standard errors could not be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Sfe/rndterd Efrmfrs fer Pm^ess Emphases 
m Mathematics, Grad® $>, 



THE NATION S 



REPORT 

CARD 




©opiate © 

In This Mathematics Class 
How Often Do 
You Address... 

Learning Mathematical 
Facts and Concepts 
Advanced Study 
Non-Eligibles 

Learning Skills and 
Procedures Needed to 
Solve Routine Problems 
Advanced Study 
Non-Eligibles 

Developing Reasoning and 
Analytical Ability to Solve 
Unique Problems 
Advanced Study 
Non-Eligibles 

Learning How to 
Communicate Ideas in 
Mathematics Effectively 
Advanced Study 
Non-Eligibles 
i 






None 


k Little 


Some 


A Lot 


1.4 


2.5 


2.3 


3.8 


0.0 


1.9 


2.6 


3.0 




1.6 


2.2 


2.7 




0.3 


2.5 


2.5 




1.7 


3.8 


4.4 


0.3 


2.0 


3.5 


3.4 




2.9 


3.1 


4.6 


0.2 


2.3 


3.3 


3.2 



- - - Standard errors cauld nat be accurately determined. 

SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 




Standard Errors fop €a kulatar Emphases 


REPORT 

CARD 




m Mathematics, ©rade 8, 1996 




4 





RartMOjp 00 ©ODIilSDaOS 




Yes 


No 


©suite © 






Do You Permit Students in This Class 






Unrestricted Use of Calculators... 






Advanced Study 


5.2 


5.2 


Non-Eligibles 


3.2 


3.2 


Do You Permit Students in This Class 






to Use Calculators for Tests... 






Advanced Study 


3.3 


3.3 


Non-Eligibles 


2.8 


2.8 


Do Students in This Class 
Have Access to Calculators 






Owned by the School... 






Advanced Study 


3.6 


3.6 


Non-Eligibles 


3.2 


3.2 


Do You Provide Instruction to 






Students in This Class in the 






Use of Calculators... 






Advanced Study 


3.8 


3.8 


Non-Eligibles 


2.7 


2.7 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 




Standard Errors fop Average Mathematics Scale 
Scares by Eligibility fop Advanced! Study, Grade 8 



THE NATION’S 






Average Scale Score 




i.i 

2.0 

1.0 


All Students 

Eligible 
Not Eligible 

i 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Percentage Correct 



@CrC3°te © 




All Students 


i.i 


Males 


1.3 


Females 


1.1 


White 


0.9 


Black 


2.3 


Hispanic 


2.1 


Asian/Pacific Islander 


1.7 


American Indian 


* * * 



### Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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mdmd Bs’p@ti’§ for Se@re Perc<srsft@g@s for 

lies/' ©nadle 8 



: THE NATION’S 
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CARO 








Correct 


Partial 


Incorrect 


Omit 












All Students 


1.2 


1.8 


1.7 


1.3 


Males 


1.6 


2.1 


1.9 


1.4 


Females 


1.2 


2.0 


2.1 


1.6 


White 


1.4 


1.9 


2.0 


1.7 


Black 


1.2! 


3.3! 


3.9! 


1.9! 


Hispanic 


2.7 


3.6 


4.0 


3.4 


Asian/Pacific Islander 


3.2 


4.4 


3.9 


3.7 


American Indian 


* * * 


* * * 


* * * 


* * * 


i 



*** Sample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Correct 


Portiol 


Incorrect 


Omit 












All Students 


1.6 


1.3 


1.8 


1.5 


Males 


2.0 


1.8 


2.2 


1.5 


Females 


1.6 


1.2 


2.1 


1.9 


White 


1.8 


1.5 


2.1 


1.8 


Black 


2.3! 


0.9! 


1.8! 


2.3! 


Hispanic 


3.5 


2.6 


4.8 


3.2 


Asian/Pacific Islander 


3.9 


2.8 


4.3 


3.4 


American Indian 


* * * 


★ * ★ 


* * * 


* * * 



C 



*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 



0 




ERIC Estimation Skills, Mathematics-in-Context, and Advanced Skills in Mathematics 



ZB-3S) 




Stitsumdaird fe>/r Sc@/r® fe/r 
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Correct 


Partial 


Incorrect 


Omit 












All Students 


1.9 


0.7 


1.8 


2.1 


Males 


2.2 


0.9 


2.4 


2.4 


Females 


2.2 


0.8 


1.9 


2.2 


White 


2.0 


0.8 


2.4 


2.5 


Black 


3.1! 


0.7! 


2.1! 


3.6! 


Hispanic 


3.2 


1.3 


3.4 


4.0 


Asian/Pacific Islander 


5.9 


1.4 


6.1 


4.2 


American Indian 


* * * 


★ ★ ★ 


* * * 


* * ★ 



*** Sample size is insufficient to permit a reliable estimate. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Correct 


Partial 


Incorrect 


Omit 












All Students 


1.3 


1.2 


1.6 


1.3 


Males 


1.7 


1.1 


2.0 


1.6 


Females 


1.3 


1.6 


2.0 


1.3 


White 


1.6 


1.2 


1.9 


1.3 


Black 


0.9! 


1.7! 


2.8! 


3.5! 


Hispanic 


3.2 


1.4 


2.9 


3.2 


Asian/Pacific Islander 


5.0 


4.2 


5.6 


• 3.6 


American Indian 


* * * 


* * * 


* * * 


* * * 



*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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IB-411 







THE NATION’S 




Sftomdlsirdl Errors §@r Stfundhsmti' 
B>®m@Gjjr<§iplhk ®Ss&r5foviQ$@m, ©rode H 2, 


REPORT 

CARD 


m®o\\ 

=H 





ffeoascaOigs ®0 SOarQarOs 


Advanced Sfudy 


Not Eligible for 
Advanced Study 


@(K3°)© 112 






Gender 






Males 


1.8 


0.9 


Females 


1.8 


0.9 


Students who Indicated Their 






Race/Ethnicity as... 






White 


2.2 


0.9 


Black 


1.0 


0.6 


Hispanic 


1.5 


0.5 


Asian/Pacific Islander 


1.6 


0.6 


American Indian 


0.1 


1.0 


Students who Reported Their 






Parents' Highest Level 






of Education as... 






Did Not Finish High School 


0.7 


0.6 


Graduated From High School 


1.0 


0.9 


Some Education After High School 


1.1 


0.9 


Graduated From College 


1.7 


1.6 


1 Don't Know 


0.3 


0.3 


Home Environment Contains... 






0-2 types of educational materials 


1.0 


0.9 


3 types of educational materials 


1.0 


0.8 


4 types of educational materials 


1.3 


1.2 


Students From... 






Northeast 


3.8 


1.4 


Southeast 


3.4 


2.3 


Central 


2.9 


1.7 


West 


3.0 


2.6 


Students Live in... 






Center City 


4.3 


3.9 


Urban Fringe/Small City 


4.9 


3.9 


Rural 


4.1 


3.0 


L 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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BS’swiidenindl fer Bffnj>d®imS‘/B<sl}ii©©ll RE card 




TfefeO© B&flQ 


B®m©gr©phh ©ssPri/bwPfl® mis, ©mdl® 12, 




t 





(Rgmifiip 0 G SfalkEfe 


Advanced Study 


Not Eligible for 
Advanced Study 


©CrQIste 






Students who Attend... 






Public Schools 


3.6 


1.2 


Nonpublic Schools 


. 3.6 


1.2 


Title 1 Participation... 






Participated 


1.0 


0.7 


Did Not Participate 


1.0 


0.7 


Free/Reduced-Price Lunch 






Program Eligibility... 






Eligible 


1.4 


1 .5 


Not Eligible 


4.5 


3.6 


Information Not Available 

i - 


4.7 


3.6 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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©irGjdl® 112, H ^^(q) 








Little 

Emphasis 


Moderate 

Emphasis 


Heavy 

Emphasis 


US 








How Much Emphasis on... 








Algebra 


1.3 


3.6 


3.3 


Geometry 


2.1 


4.1 


3.6 


Trigonometry 


1.4 


2.8 


2.7 


Functions 


0.8 


2.1 


2.1 


Statistics 


2.9 


2.7 


1.9 


Probability 


2.4 


2.6 


2.1 


Discrete Mathematics 
1 — — 


3.2 


3.1 


1.4 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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[fojsaiiiG^ 


Little 

Emphasis 


[Moderate 

Emphasis 


Heavy 

Emphasis 


(cfosdfe US 








How Much Emphasis on... 








Learning Mathematical 








Facts and Concepts 


0.9 


3.3 


3.3 


Learning Skills and Procedures 








Needed to Solve Routine Problems 


0.4 


3.1 


3.2 


Developing Reasoning and 








Analytical Ability to Solve 








Unique Problems 


1.0 


2.9 


3.1 


Learning How to Communicate 








Ideas in Mathematics Effectively 

i 


1.9 


3.7 


3.5 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Percentage of Students 






Do You Permit Students in This Class 




Unrestricted Use of Calculators... 




Yes 


2.4 


No 


2.4 


Do Students in This Class 




Have Access to Scientific Calculators 




in Class... 




Yes 


2.8 


No 


2.8 


Do Students in This Class 




Have Access to Scientific Calculators 




Out of Class... 




Yes 


2.6 


No 


2.6 


Do Students in This Class 




Have Access to Graphing Calculators 




in Class... 




Yes 


3.0 


No 


3.0 


Do Students in This Class 




Have Access to Graphing Calculators 




Out of Class... 




Yes 


3.4 


No 


3.4 


Do You Permit Students in This Class 




the Use of Calculators on Tests... 




' Yes All 


3.3 


Yes Some 


3.3 


No 

r 


0.1 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1 996 
Mathematics Assessment. 
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Sffanfifore? Ermrs for Cofewfafor &fe<s©/e <§wvdl BmUmeffmmi report 




Cfosses fo&ero by ^\d!v<siinisedl Sfttuidly BfttmdleoTiQs, CARD 


U 'ttUL 


nr 1 

i 




V2, IW 





Percentage of Students 


@R3°fe U2 




In This Class Students are 




Provided Instruction in the Use of 




Scientific Calculators... 




Yes 


3.0 


No 


3.0 


In This Class Students are 




Provided Instruction in the Use of 




Graphing Calculators... 




Yes 


2.6 


No 


2.6 


Which of the Following Best Describes 




the Availability of Graphing Calculators 




in This Class... 




One 


0.8 


Less than Six 


1.0 


Complete Set 


4.0 


Some Students Have One 


1.9 


Most Students Have One 


2.3 


All Students Have One 


3.5 


No Student Has One 


.17 


If Graphing Calculators are Used 




in This Class, What is Their 




Primary Usage... 




Calculating 


1.8 


Graphing 


3.0 


Tables 


0.2 


Statistics 


0.5 


Symbolic Manipulation 


1.4 


Not Used 


2.8 


i — 



SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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IB-47 





' THE NATION’S 


tfefcfe moW 


Si?@in)(s7@/?-(s7 lEsrmsrs (fair $n?er@§j® M©dhem©(i , ks §<s@l@ 


§e@ir®$ by Skgmmty (far Adhramed Btiwdy, ©md® H2 — 


1: 







Average Scale Score 


tl2 








All Students 


1.0 




Eligible 


1.6 


1 


Not Eligible 


0.9 



SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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THE NATION’S 
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CARD 






Percentage Correct 


@K°X°)3 tl2 




All Students 


0.4 


Males 


1.0 


Females 


0.9 


White 


0.8 


Black 


1.6 


Hispanic 


1.4 


Asian/Pacific Islander 


2.2 


American Indian 


* * * 


Are Students Presently Enrolled 




in Mathematics... 




^ Yes 


0.8 


No 


1.4 


Are Students Presently Enrolled in or 




Have They Previously Taken an 




Advanced Placement (AP) 




Mathematics Course... 




Yes 


1.1 


No 

l 


0.9 

— _ — — , * ■ - - £ - — — _ — 



*** Sample size is insufficient to permit a reliable estimate. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Lion®© ir IFme95@i n," ©m<d® H2 



y ty§® 



“ THE MAHON’S 

BE SR 






Correct 


Partial 


Incorrect 


Omit 












All Students 


1.1 


0.6 


1.2 


1.3 


Males 


1.3 


0.9 


1.5 


1.6 


Females 


1.3 


0.9 


1.7 


1.4 


White 


1.2 


0.7 


1.4 


1.3 


Black 


1.2 


1.7 


3.3 


3.4 


Hispanic 


2.4! 


1.7! 


3.5! 


3.2! 


Asian/Pacific Islander 


4.2 


1.7 


4.0 


2.6 


American Indian 


* ★ ★ 


* ★ ★ 


★ * * 


* * * 


Are Students Presently 










Enrolled in Mathematics... 

Yes 


1.1 


0.7 


1.2 


1.3 


No 


3.2! 




5.6! 


5.2! 


Are Students Presently 
Enrolled in or Have 
They Previously Taken an 










Advanced Placement (AP) 










Mathematics Course... 










Yes 


1.9 


0.9 


1.6 


1.6 


No 


1.4 


0.9 


1.7 


1.7 



[ 



*** Sample size is insufficient ta permit a reliable estimate. 

— Standard errors cauld nat be accurately determined. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may nat be accurately 
determined and/ar the sampling distribution of the statistics does nat match statistical test assumptions (see Appendix A). 
SOURCE: National Center far Education Statistics, National Assessment af Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Emroirs {?©ir Pems/ratagjes fo/r 

"€©mpm<& V©S(wm(B5 ©if Pyir@miidl§, "<&mde 12 



THE NATION’S 






(Extended 


Satisfactory 


Partial 


fUlinimal 


Incorrect 


Omit 
















All Students 


0.5 


0.5 


0.7 


0.9 


1.2 


1.2 


Males 


0.7 


0.8 


0.8 


1.1 


1.5 


1.7 


Females 


0.6 


0.7 


0.7 


1.1 


1.8 


1.3 


White 


0.5 


0.6 


0.8 


1.2 


1.3 


1.2 


Black 




— 


0.6 


1.5 


5.4 


5.2 


Hispanic 


1.2! 


0.7! 


0.5! 


1.5! 


5.1! 


4.5! 


Asian/Pacific Islander 


1.9 


1.7 


2.0 


1.6 


2.5 


2.4 


American Indian 


★ ★ ★ 


* * * 


* * * 


★ ★ ★ 


★ ★ ★ 


★ ★ ★ 


Are Students Presently 














Enrolled in Mathematics... 














Yes 


0.5 


0.5 


0.7 


0.9 


1.3 


1.2 


No 


1.11 




0.4! 


4.4! 


5.9! 


5.1! 


Are Students Presently 














Enrolled in or Have 














They Previously Taken an 
Advanced Placement (AP) 














Mathematics Course... 














Yes 


1.1 


0.9 


0.9 


1.3 


1.7 


1.3 


No 


0.5 


0.6 


0.9 


1.2 


1.6 


1.7 


L 



*** Sample size is insufficient to permit a reliable estimate, 

- - - Standard errors could not be accurately determined. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Percentage of Students 


US 




All Students 


1.0 


Males 


1.5 


Females 


1.3 


White 


1.1 


Black 


4.1 


Hispanic 


3.3! 


Asian/Pacific Islander 


3.1 


American Indian 


* * * 


Are Students Presently 
Enrolled in Mathematics... 




Yes 


1.0 


No 


4.2 


Are Students Presently Enrolled in or 
Have They Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 




Yes 


1.7 


No 


1.2 


1 





*** Sample size is insufficient to permit a reliable estimate. 

! Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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Expended 


Satisfactory 


Partial 


(Minimal 


Incorrect 


Omit 
















All Students 


1.5 


0.1 


0.9 


0.8 


1.2 


0.9 


Males 


1.9 


0.3 


1.1 


1.1 


1.3 


1.1 


Females 


1.7 


... 


1.3 


1.1 


1.6 


1.0 


White 


1.6 


0.2 


1.1 


0.9 


1.4 


0.8 


Black 


2.9 


— 


2.8 


2.3 


2.8 


4.3 


Hispanic 


4.1! 


— 


1.7! 


2.0! 


2.7! 


3.1! 


Asian/Pacific Islander 


3.2 


— 


2.4 


2.2 


2.7 


2.3 


American Indian 


* * * 


* * * 


★ ★ ★ 


* * * 


* * * 


* * * 


Are Students Presently 
Enrolled in Mathematics... 














Yes 


1.5 


0.1 


0.9 


0.8 


1.2 


1.0 


No 


4.2! 




3.7! 


4.3! 


4.0! 


3.3! 


Are Students Presently 
Enrolled in or Have 
They Previously Taken an 
Advanced Placement (AP) 
Mathematics Course... 














Yes 


1.7 


— 


1.4 


1.0 


1.1 


1.1 


No 


2.1 


0.2 


1.1 


1.2 


1.6 


1.1 



[ 



*** Sample size is insufficient to permit a reliable estimate. 

--- Standard errors could not be accurately determined. 

I Statistical tests involving this value should be interpreted with caution. Standard error estimates may not be accurately 
determined and/or the sampling distribution of the statistics does not match statistical test assumptions (see Appendix A). 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1996 
Mathematics Assessment. 
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